ASEN 6519 - DMU ++ Paper Presentation on "PAWS"

Himanshu Gupta

Date - 18 September 2023

Fang et al. (AAMAS 2017)

MOTIVATION

• Poaching is the illegal hunting or capturing of wild animals, usually associated with land use rights


 
• Poaching can lead to disastrous consequences
  • Population loss
  • Unbalanced Ecosystem
  • Spread of major illnesses 

Source

Source

MOTIVATION

• So, how can we prevent/reduce poaching?
   
• Patroling is the most common and effective preventive measure at the moment.
   
• But figuring out the best patrolling routes to minimize poaching over a large-scale conservation area is difficult.
  • Limited patroling resources
  • Can AI help?

MOTIVATION

• PAWS-Initial (Yang et al. 2014) used a game theoretic formulation to solve this problem.
  • Protection Assistant for Wildlife Security
  • Why the game theoretic approach?
  • Has 4 major limitations
    • So, now what?

CONTRIBUTION

• Developed PAWS, a more realistic and practical game-theoretic application deployed in Southeast Asia for optimizing foot patrols to combat poaching.
   
• Addressed major limitations of its predecessor, PAWS-Initial.
  • PAWS is a regularly deployed application, while PAWS-Initial was a proposed decision aid.
     
• Patrollers and patrol planners agreed that PAWS generates detailed suggested routes that can guide the actual patrol.
  • Feedback: Suggested routes were easier to follow, compared with the routes from PAWS-Initial.
  • Reduces/Removes the mental effort of planning routes. 

Theory

• Stackelberg game: One player (the "leader") moves first, and other players ("the followers") observe their move and move after them.

Theory

• Stackelberg game: One player (the "leader") moves first, and other players ("the followers") observe their move and move after them.

Theory

• Stackelberg game: One player (the "leader") moves first, and other players ("the followers") observe their move and move after them.

Theory

• Minimax regret game - a game where the best strategy is the one that minimizes the maximum regret. Essentially, this is the technique for a 'sore loser' who does not wish to make the wrong decision.

Payoff Table

Regret

PAWS-Initial

• Solves a Stackelberg security game (SSG)
  • Discretize conservation area into a grid.
  • \(U^a_{p,i},U^a_{r,i},U^d_{p,i},U^d_{r,i} \)
     
• Defender strategy can be represented using a coverage vector \(c\).
  • \(<c_i>\) - probability defender goes to target i
    
     
• Given a defender strategy \(c\) and the penalty and reward values, the players’ expected utilities \(U^a_i\) and \(U^d_i\) when target i is attacked can be calculated.

PAWS-Initial LIMITATIONS

1. Ignored topographic information

PAWS-Initial LIMITATIONS

1. Ignored topographic information (go through a lake)

PAWS-Initial LIMITATIONS

1. Ignored topographic information (too much gradient)

PAWS-Initial LIMITATIONS

1. Ignored topographic information
    
2. Assumes payoff values of the targets, \(U^a_{r,i}\) are known and fixed
   • Animals can migrate from season to season or depending on food availability.
      
3. Doesn't scale to large areas.
   • Finer discretization is needed but that makes the problem exponential.
      

PAWS-Initial LIMITATIONS

4. Considers covering individual grid cells, but not feasible paths.

So, solution?

Approach

Input and Initial Analysis

• Produce a posterior predictive for animals using a Bayesian framework.
   
• MaxEnt for human activity distribution
  • using geographical data, e.g., distance to water, slope, elevation.

Building Game Model

• Based on the input information and the estimated distribution, build a game model
  • Use SSG to abstract the strategic interaction between the patroller and the poacher.
     
• Building a game model involves
  • defender action modeling
  • adversary action modeling
  • payoff modeling.

Building Game Model

• Defender Action Modeling
  • Patroller's feedback - detailed guidance is needed!
     
  • If they use a fine-grained grid where every grid cell is a target, computing the optimal patrolling strategy is computationally challenging (PROBLEM)
     
  • Hierarchical modeling approach (SOLUTION)
    • allows to attain a good compromise between scaling up and providing detailed guidance
    • (1km x 1km Grid cells and 50m x 50m Raster pieces)
    • Defender actions are patrol routes defined over a virtual “street map”

Building Game Model

• Defender Action Modeling (Hierarchical modeling approach)
     
  • Generate a street map
    • A graph consisting of nodes and edges, where the set of nodes is a small subset of the raster pieces and edges are sequences of raster pieces linking the nodes.
    • Nodes are Key Access Points (KAPs) and edges as route segments.

Building Game Model

• Defender Action Modeling:
  • Building a street map (3 steps)
  • Step 1: determine the accessibility type for each raster piece
  • Step 2: define KAPs
  • Step 3: find route segments to link the KAPs.

Poacher Action Modeling 

• To ensure scalability, poacher’s actions are defined over the coarse grid cells.
   
• In this game, the assumption is that the poacher can observe the defender’s mixed strategy and then choose a grid cell to attack.
   
• Poacher is assumed to be boundedly rational, and their actions can be described by the SUQR model.
  • A weighted combination of coverage probability, the poacher's reward and the poacher's penalty

Payoff Modeling 

• Models a zero-sum game
  • reward for attacker = penalty for defender
     
• Attacker's reward at a target (grid cell) is decided by the animal distribution.
   
• However, animal density is difficult to determine exactly and can vary due to seasonal or dynamic migration.
  • This makes the payoff uncertain.
  • Need to handle uncertainty in the players’ payoff values.
     
• Use intervals to represent payoff uncertainty in PAWS.
  • If the patrollers patrol a cell more frequently, there is less uncertainty in the poacher’s payoffs at that target and thus a smaller size of the payoff intervals.

Calculate Patrol Strategy

• The algorithm should
   
  • Generate patrol routes over the street map over the entire conservation area region
     
  • Address payoff uncertainty
     
  • Work with bounded rationality of the adversary (SUQR)
    • instead of assuming a completely rational agent.

Calculate Patrol Strategy

• Solution: ARROW with BLADE
     
  • ARROW finds the strategy that minimizes regret  for the patrollers over the coarse grid cells.
     
  • BLADE makes it scalable

 Constraint: \( c_1 \geq (c_2 + c_3 + c_4) \)

Calculate Patrol Strategy

Calculate Patrol Strategy

• PAWS calculates the patrol strategy consisting of a set of patrol routes and the corresponding probability for taking them.

Trading Off Exploration and Exploitation

• PAWS-EvE
    
  • Offers the option of assigning a probability range for selecting an explorative route.
     
  • Explorative routes cover a significant portion of previously unpatrolled land while Exploitative routes cover a significant portion of land previously patrolled.
     
  • Exploitative routes are great, but since the objective of PAWS is to minimize poaching activity, it is necessary to also take explorative routes.

Deployment and Evaluation

• PAWS patrols are now regularly deployed at a conservation area in Malaysia.
  • Daily patrols from base camps.
  • 10 km per day distance limit.
    
    
    
    
    
     
  • Impossible to consider each raster piece as a separate target or consider all possible routes over the raster pieces.
  • Hierarchical approach: 8.57(= 194.33/22.67) KAPs and 80 route segments in each grid cell on average

Deployment and Evaluation

• PAWS patrol lasts for 4-5 days and is executed by a team of 3-7 patrollers.
   
• During the patrol, they detect animal and human activity signs and record them with detailed comments and photos.
   
• After the patrol, the data manager will put all the information into a database, which helps refine the map and information for future implementations of PAWS.

Deployment and Evaluation

Deployment and Evaluation

Deployment and Evaluation

PAWS-Initial

PAWS

Critique and Limitations

• Motivation for why the game theoretic approach is the best way to solve this problem?
  • Why is the minimax regret game formulation better than SSG?
  • Different discretization for poacher and patroller
     
• The paper has almost no math in it. How does ARROW find the patrol strategy?
  • Maybe that is also good?
     
• No qualitative results that show PAWS works better than PAWS-Initial. 
  • Same for PAWS-EvE

Impact and Legacy

• The paper came out in 2017 and has 68 citations.
  • PAWS-Initial came out in 2014 and has 235 citations.
     
• A fascinating real-life application for game theoretic approaches.
  • Opened the path for other security related real-life applications, like patroling of large warehouses or college campuses.

Future Work

• They didn't mention any in their paper.
  • Other than expanding it to other large conservation areas in other countries.
    
     
• Can be formulated using the Dec-POMDP framework. 
  • Comparing the two approaches to see which works better is a good research question we don't know the answer to.
  • We have a paper on Dec-POMDPs towards the end of the class.

CONTRIBUTION

• Developed PAWS, a more realistic and practical game-theoretic application deployed in Southeast Asia for optimizing foot patrols to combat poaching.
   
• Addressed major limitations of its predecessor, PAWS-Initial.
  • PAWS is a regularly deployed application, while PAWS-Initial was a proposed decision aid.
     
• Patrollers and patrol planners agreed that PAWS generates detailed suggested routes that can guide the actual patrol.
  • Suggested routes were easier to follow, compared with the routes from PAWS-Initial.
  • PAWS was able to guide them towards poaching hotspots.
  • Reduces/Removes the mental effort of planning routes.