Northerners thought that emancipation and arming the blacks would create “a more terrible [and] effective weapon against the Southerners,” alluding to the result of a Santo Domingo-style race war in the South. In reality, black men were enticed off plantations to deny the agricultural South its laborers, who were then recruited into regiments to serve as lowly paid laborers and servants.
Bernhard Thuersam, www.circa1865.org
Few Black Volunteers at Hilton Head
“The man chosen to fill the office [military governor at Hilton Head] was Rufus B. Saxton, a newly-breveted brigadier general. A native of Deerfield, Maine, a graduate of West Point and a career officer in the army, Saxton came to Hilton Head with the assault force as a captain in the Quartermaster Corps. His father had been an enthusiastic abolitionist, yet Saxton took his assignment with reluctance and out of loyalty rather than out of sympathy for the Negro.
In the summer of 1862, Laura Towne, one of the Northern teachers on St. Helena, was pleased to find him “truly anti-slavery.” Major General David M. Hunter [at that time served on Hilton Head as], commanding general of the Department of the South.
Hunter, acting on his own authority . . . forced the issue by beginning the recruitment of a regiment of Negro soldiers. [He] was able to muster 150 Negroes into the service as the First South Carolina Volunteers. Thereafter, however, recruiting proceeded slowly. Most of the volunteers probably were refugees from the mainland without employment [and] those who remained on the plantations and were engaged in planting their crops were far from enthusiastic.
On St. Helena, it was reported that only one man volunteered, and the missionaries generally agreed that the Negroes were afraid of “being made to fight.” On St. Helena, Laura Towne observed that the plantation hands generally regarded the maneuver as “a trap to get the able-bodied and send them to Cuba to sell . . .” Miss Towne asserted that “nearly all are eager to go there again and serve in the forts,” but they did not want to fight.
Whereas many Negroes volunteered “willingly” in the first few days of Saxton’s recruiting campaign, some offered themselves with “dismal forlornness,” and others not at all.
When two officers appeared at a church on St. Helena on October 23  to seek recruits, all able-bodied males declined to attend. On the following Sunday, Sergeant Prince Rivers, a Negro veteran of the Hunter Regiment, visiting the island on the same mission, suffered the same disappointment.”
(After Slavery, The Negro in South Carolina During Reconstruction, Joel Williamson, UNC Press, 1965, pp. 13-17)