mid-15c., "wield a tool," also, late 15c., "to attack (an enemy)," from man (n.) + handle (v.). Nautical meaning "to move by force of men" (without levers or tackle) is attested from 1834, and is the source of the slang meaning "to handle roughly" (1865).
[T]he two Canalers rushed into the uproar, and sought to drag their man out of it toward the forecastle. Others of the sailors joined with them in this attempt, and a twisted turmoil ensued; while standing out of harm's way, the valiant captain danced up and down with a whale-pike, calling upon his officers to manhandle that atrocious scoundrel, and smoke him along to the quarter-deck. [Melville, "The Town-Ho's Story," "Harper's" magazine, October 1851]
