Mega Followers maxes out very quickly, you need to get the points on the map after its done. You can only max out maybe a little more than half of your skills with mega followers, imo.
EDIT: You stop gaining points after a certain amount of followers, if that was not clear...