By recording single neurons across both tasks, we show that the discount factor at the single neuron level is correlated across tasks suggesting that discounting is a cell-specific property, constraining implementations of dopamine-based multi-timescale RL.

Next, we show that heterogeneity in ramping activity across dopamine neurons when mice approach a reward in a 1-D VR track can also be understood as a signature of diverse discount factors across neurons.

First, we quantify the discount factor by measuring the relative response to cues predicting rewards at different delays and show that reward timing information can be decoded from the vectorized prediction error.

We first explore four computational advantages of artificial multi-timescale RL agents:
- disentangling reward timing and reward magnitude
- learn values with non-exponential temporal discounts
- infer temporal information before convergence
- implement a state-dependent discount factor