Well, I spent a long time writing a details post with an example, but ran into the 2000 character limit. So I'll just say:
In the context of computing derivatives, e is an arbitrary number that is close to zero. The dual numbers model some things we do with e, and that is one motivation for them. In the context of the dual numbers, e is just a symbol that we define in a certain way, and nothing more.
Edit: I guess I can add one more thing. One of the things we do with e is compute derivatives. The definition of the derivative is:
f'(x) = lim_e->0 [ f(x + e) - f(x) ] / e
We can't just set e = 0 here because then we're dividing by zero. Instead, we let e get closer and closer to 0 to see what the limit tends to. However, we can "set" e^2 = 0 here to simplify some calculations. The dual numbers rigorously model this for us.