Ok. Let A = {1,2,3,4,5}=B. We are looking for functions f: A → B satisfying f(f(x)) = f(x).
Now for the sake of exploration, you see what happens if say f(1) =2.
Then f(f(1)) = f(2) = 2. By this it is clear that if 2 belongs to the range of f, then we must have f(2) = 2. You can extend this to say that this is true of any number a that appears in the range, i.e. f(a) = a.
If the function is onto then it is clear from the above discussion that the only function possible is f(x) = x.
What if its not onto? Let us say that only 1,4, and 5 appear in the range. I have conveyed this same idea as "1,4, and 5 have pre-images under f".
Its clear that f(1) =1, f(4) = 4, f(5)=5.
Now the question is how to assign f(2) and f(3) in such a way that the condition f(f(x)) = f(x) still holds?
Again, some exploration. Say f(2) =1. Then all that follows is that f(f(2)) = f(1) =1 and all izz well. That means f(2) can be randomly chosen from among {1,4,5}. The same thing holds true for f(3).
Now we have our strategy in place to construct any such function.
(1) Pick how many numbers appear in the range. Call this k. k can be 1,2,3,4 or 5. If k=5, the function is onto.
(2) Pick the k numbers that appear in the range. Let them be x1,...,xk
Then f(xk) = xk
(3) The remaining numbers i.e. xk+1,...,x5 may be be randomly assigned to {x1,...,xk} i.e. the remaining (5-k) numbers each have a choice of k numbers to be assigned to.
Its obvious that every such function is obtained by this procedure
Now, we only need to count how many functions are obtained this way.
The selection of the k numbers that appear in the range is 5Ck
The remaining (5-k) numbers have k valid choices each, which means it can independently be assigned in k5-k ways.
Then add up for each k i.e. k=1,2,3,4,5.