The Internet Group Management Protocol (IGMP) is the protocol used for router multicasting, allowing hosts to report multicast group memberships to neighboring routers. Other options mentioned, such as ICMP and RIP, serve different functions and are not used for multicast management. Security issues are noted with IGMP, as it allows unrestricted creation and joining of multicast groups.