PAWS-Initial (Yang et al. 2014) used a game theoretic formulation to solve this problem.
Protection Assistant for Wildlife Security
Why the game theoretic approach?
Has 4 major limitations
So, now what?
CONTRIBUTION
Developed PAWS, a more realistic and practical game-theoretic application deployed in Southeast Asia for optimizing foot patrols to combat poaching.
Addressed major limitations of its predecessor, PAWS-Initial.
PAWS is a regularly deployed application, while PAWS-Initial was a proposed decision aid.
Patrollers and patrol planners agreed that PAWS generates detailed suggested routes that can guide the actual patrol.
Feedback: Suggested routes were easier to follow, compared with the routes from PAWS-Initial.
Reduces/Removes the mental effort of planning routes.
Theory
Stackelberg game: One player (the "leader") moves first, and other players ("the followers") observe their move and move after them.
Theory
Stackelberg game: One player (the "leader") moves first, and other players ("the followers") observe their move and move after them.
Theory
Stackelberg game: One player (the "leader") moves first, and other players ("the followers") observe their move and move after them.
Theory
Minimax regret game - a game where the best strategy is the one that minimizes the maximum regret. Essentially, this is the technique for a 'sore loser' who does not wish to make the wrong decision.
Payoff Table
Regret
PAWS-Initial
Solves a Stackelberg security game (SSG)
Discretize conservation area into a grid.
\(U^a_{p,i},U^a_{r,i},U^d_{p,i},U^d_{r,i} \)
Defender strategy can be represented using a coverage vector \(c\).
\(<c_i>\) - probability defender goes to target i
Given a defender strategy \(c\) and the penalty and reward values, the players’ expected utilities \(U^a_i\) and \(U^d_i\) when target i is attacked can be calculated.
PAWS-Initial LIMITATIONS
Ignored topographic information
PAWS-Initial LIMITATIONS
Ignored topographic information (go through a lake)
PAWS-Initial LIMITATIONS
Ignored topographic information (too much gradient)
PAWS-Initial LIMITATIONS
Ignored topographic information
Assumes payoff values of the targets, \(U^a_{r,i}\) are known and fixed
Animals can migrate from season to season or depending on food availability.
Doesn't scale to large areas.
Finer discretization is needed but that makes the problem exponential.
PAWS-Initial LIMITATIONS
4. Considers covering individual grid cells, but not feasible paths.
So, solution?
PAWS!
Approach
Input and Initial Analysis
Produce a posterior predictive for animals using a Bayesian framework.
MaxEnt for human activity distribution
using geographical data, e.g., distance to water, slope, elevation.
Building Game Model
Based on the input information and the estimated distribution, build a game model
Use SSG to abstract the strategic interaction between the patroller and the poacher.
Building a game model involves
defender action modeling
adversary action modeling
payoff modeling.
Building Game Model
Defender Action Modeling
Patroller's feedback - detailed guidance is needed!
If they use a fine-grained grid where every grid cell is a target, computing the optimal patrolling strategy is computationally challenging (PROBLEM)
Hierarchical modeling approach (SOLUTION)
allows to attain a good compromise between scaling up and providing detailed guidance
(1km x 1km Grid cells and 50m x 50m Raster pieces)
Defender actions are patrol routes defined over a virtual “street map”
A graph consisting of nodes and edges, where the set of nodes is a small subset of the raster pieces and edges are sequences of raster pieces linking the nodes.
Nodes are Key Access Points (KAPs) and edges as route segments.
Building Game Model
Defender Action Modeling:
Building a street map (3 steps)
Step 1: determine the accessibility type for each raster piece
Step 2: define KAPs
Step 3: find route segments to link the KAPs.
Poacher Action Modeling
To ensure scalability, poacher’s actions are defined over the coarse grid cells.
In this game, the assumption is that the poacher can observe the defender’s mixed strategy and then choose a grid cell to attack.
Poacher is assumed to be boundedly rational, and their actions can be described by the SUQR model.
A weighted combination of coverage probability, the poacher's reward and the poacher's penalty
Payoff Modeling
Models a zero-sum game
reward for attacker = penalty for defender
Attacker's reward at a target (grid cell) is decided by the animal distribution.
However, animal density is difficult to determine exactly and can vary due to seasonal or dynamic migration.
This makes the payoff uncertain.
Need to handle uncertainty in the players’ payoff values.
Use intervals to represent payoff uncertainty in PAWS.
If the patrollers patrol a cell more frequently, there is less uncertainty in the poacher’s payoffs at that target and thus a smaller size of the payoff intervals.
Calculate Patrol Strategy
The algorithm should
Generate patrol routes over the street map over the entire conservation area region
Address payoff uncertainty
Work with bounded rationality of the adversary (SUQR)
instead of assuming a completely rational agent.
ARROW
(Nguyen et al. 2015)
Calculate Patrol Strategy
Solution: ARROW with BLADE
ARROW finds the strategy that minimizes regret for the patrollers over the coarse grid cells.
BLADE makes it scalable
Constraint: \( c_1 \geq (c_2 + c_3 + c_4) \)
Calculate Patrol Strategy
Calculate Patrol Strategy
PAWS calculates the patrol strategy consisting of a set of patrol routes and the corresponding probability for taking them.
Trading Off Exploration and Exploitation
PAWS-EvE
Offers the option of assigning a probability range for selecting an explorative route.
Explorative routes cover a significant portion of previously unpatrolled land while Exploitative routes cover a significant portion of land previously patrolled.
Exploitative routes are great, but since the objective of PAWS is to minimize poaching activity, it is necessary to also take explorative routes.
Deployment and Evaluation
PAWS patrols are now regularly deployed at a conservation area in Malaysia.
Daily patrols from base camps.
10 km per day distance limit.
Impossible to consider each raster piece as a separate target or consider all possible routes over the raster pieces.
Hierarchical approach: 8.57(= 194.33/22.67) KAPs and 80 route segments in each grid cell on average
Deployment and Evaluation
PAWS patrol lasts for 4-5 days and is executed by a team of 3-7 patrollers.
During the patrol, they detect animal and human activity signs and record them with detailed comments and photos.
After the patrol, the data manager will put all the information into a database, which helps refine the map and information for future implementations of PAWS.
Deployment and Evaluation
Deployment and Evaluation
Deployment and Evaluation
PAWS-Initial
PAWS
Critique and Limitations
Motivation for why the game theoretic approach is the best way to solve this problem?
Why is the minimax regret game formulation better than SSG?
Different discretization for poacher and patroller
The paper has almost no math in it. How does ARROW find the patrol strategy?
Maybe that is also good?
No qualitative results that show PAWS works better than PAWS-Initial.
Same for PAWS-EvE
Impact and Legacy
The paper came out in 2017 and has 68 citations.
PAWS-Initial came out in 2014 and has 235 citations.
A fascinating real-life application for game theoretic approaches.
Opened the path for other security related real-life applications, like patroling of large warehouses or college campuses.
Future Work
They didn't mention any in their paper.
Other than expanding it to other large conservation areas in other countries.
Can be formulated using the Dec-POMDP framework.
Comparing the two approaches to see which works better is a good research question we don't know the answer to.
We have a paper on Dec-POMDPs towards the end of the class.
CONTRIBUTION
Developed PAWS, a more realistic and practical game-theoretic application deployed in Southeast Asia for optimizing foot patrols to combat poaching.
Addressed major limitations of its predecessor, PAWS-Initial.
PAWS is a regularly deployed application, while PAWS-Initial was a proposed decision aid.
Patrollers and patrol planners agreed that PAWS generates detailed suggested routes that can guide the actual patrol.
Suggested routes were easier to follow, compared with the routes from PAWS-Initial.
PAWS was able to guide them towards poaching hotspots.
Reduces/Removes the mental effort of planning routes.