Motivation

• Forests have are important sources for building materials (historically for fuel)
• Biodiversity, carbon capture, are becoming more important factors
• Managing involves reasoning about costs, rewards, growth dynamics, fires, etc.
• Previous state-of-the-art-optimization was not scalable

Contributions

• Bringing Reinforcement Learning to Forest management!
  • Biggest stochastic forestry optimization problem ever solved
  • Drastically reduced computation time
  • No need to simplify models
• Demonstrated that RL converges to previously-known optimal strategy
• Discovered new strategies when the uncertainty of disasters is included
• Explained why some results were expected and some unexpected with a classic principle

Forestry 101

• Four tree species:
  • Norway Spruce
  • Silver Birch
  • Scots Pine
  • European Aspen
• CCF = Continuous Cover Management
• RF = Rotation Forestry
• Central Finland
• Disasters

Context in Literature

• \(j\) = species
• \(s\) = size class
• \(\tilde{x}_{j,s,t}\) = number of trees
• \(\alpha_{j,s}(\tilde{x}_t)\) = fraction to new size class
• \(\mu_{j,s}(\tilde{x}_t)\) = death fraction
• \(\phi_j(\tilde{x}_t) \geq 0\) = ingrowth rate
• \(\Delta\) = period length (5 years)
• \(r\) = interest rate (discount \(b = 1/(1+r)\))
• \(h\) = number of harvested trees
• \(\pi\) = profit
• \(w\) = "cost of artificial regeneration"

Important Previous Work:

• Faustmann, 1849: Basic optimal rotation problem
• Reed, 1984: Disasters
• Parkatti and Tahvonen, 2020: Nonlinear Programming

Problem Setting

• State Space: 44-dimensional continuous
• Action Space: 44 continuous dimensions, 2 binary choices

Problem Setting

Approach

• Hybrid PPO with GAE

Computational Resources

Numerical Results: Deterministic Model

Numerical Results: Initial Clearcut?

Numerical Results:
Initial Clearcut?

Numerical Results:
Stochastic Outcomes with No Disasters

Numerical Results:
Stochastic Outcomes with No Disasters

Mean: 2357 Euros

Mean: 2355 Euros

Deterministic Mean:

2336 Euros

Numerical Results:
Stochastic Outcomes with Disasters

0% Fire

1% Fire

\(r\) = 1%

2% Fire

Disaster Probability = 1%

Disaster Probability = 2%

Policy optimized with disasters

Policy optimized with no disasters

Mean 15,151

Mean 14,753

Mean 11,101

Mean 10,060

Numerical Result Summary

• "Higher interest rate favors CCF"
• Birch is sometimes dominant
• With no disasters, similar value to deterministic
• With disasters, RL significantly better

• Confirmed Previous
• Previously Unknown
• Confirmed Previous
• New Result!

Critique

Impact and Legacy

18 citations since 2021 = Pretty Good!

Contributions (Recap)

• Brought Reinforcement Learning to Forest management!
  • Biggest stochastic forestry optimization problem ever solved
  • Drastically reduced computation time (170 h -> 6 min)
  • No need to simplify models (Could include more stochasticity)
• Demonstrated that RL converges to previously-known optimal strategy
• Discovered new strategies when the uncertainty of disasters is included
• Explained why some results were expected and some unexpected with a classic principle: Certainty Equivalence