Reward when bullet hits target

[aieie89]

Hello,
I trained an agent to reach a target. I give to the agent instant reward when he get closer to the target.
Now I added to the agent the ability to shoot "slow" bullets, and I want to give reward when one or more bullets hit the target.

So, there are two issue:

1. How can I assign reward to an agent if the event occurs between bullet and target?
2. How can agent know this reward is related to past decision? (The decision to shoot)

For the first issue, I think to instantiate bullets as childs of the agent, in order to call the parent function giveReward when the event is triggered.
For the second issue, I read something about recurrent neural network, but I don't know if it is the right way. Another possibility is to observation vector some information about the shooted bullet, in order to understand which bullets hitted?

Thanks in advance for the discussion.

[ervteng]

I'm not sure I understand your question for 1. You'd call AddReward() when it hits the target, no?

For 2, it is possible for RL algorithms to learn long-term credit assignment, though it is harder to learn than immediate rewards. How many steps are between firing and receiving the reward? You may have to increase your gamma parameter; Stacked vectors should help as well.

[aieie89]

For the first point:
yes, but I'm little confused on an aspect. My future goal is to train a multi-agent environment, with two team fighting together. So in my mind, when a bullet hit a target (or another agent), the reward goes to the agent that shooted the bullet. It's right to think on the reward as reward of a single agent? Or it has to be a global reward?

For second point:
So, steps between shoot and hit can change, depends on the size of the map. In my first little training i think the bullet hit the target in 20-30 steps. Interesting, I missed the gamma parameter, I think adjusting this can be a good try. I'm trying the "slow" bullet choise to test the other agent's ability to avoid incoming bullets. ( It's an open possibility, not sure it could be practical )

Thanks for your interest.

[xiaomaogy]

@ervteng

[aieie89]

With stacked vector = 5 and gamma = 0.99 agents learn well shooting each other.

Thank you,
for the point of multi-agent i'll open a new discussion.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.