He was generally recognized as one of the fastest players of his day. His speed was said to be compared to that of Cool Papa Bell, a star of the Negro Leagues of the 1930s, and Ty Cobb. He was a left-handed batter with a noted eye, who hit for exceptionally high batting averages. He started playing organized Negro ball for the Harrisburg Colored Giants in 1906 and first became a professional for Sol White's Philadelphia Giants in 1909. Poles soon followed White to the New York Lincoln Giants in 1911, where he blossomed into a star; in his first four seasons with the Lincoln Giants, 1911–1914, Poles attained batting averages of .440, .398, .414, and .487 against all levels of competition. Poles then spent the next few seasons jumping among the New York Lincoln Stars, Brooklyn Royal Giants, and the Hilldale Daisies. While Poles was with the Daisies, he joined the Army 369th Infantry Regiment (Harlem Hellfighters), attached to the French Army, to serve during World War I, earning decorations (five battle stars and a Purple Heart) for his combat experience in France as a sergeant. He returned home and continued a successful baseball career, playing for the Lincoln Giants from 1919 to 1923.
He is credited with a lifetime batting average of over .400 against all competition, and hit .319 in four winters in Cuba. A tantalizing aspect of his career is his success against white major league teams. Poles hit .610 against these teams, including three consecutive hits off Grover Cleveland Alexander. Poles spent his post-baseball years as a taxi cab operator and working at Olmsted Air Force Base in Middletown, Pennsylvania, enabling him to retire comfortably. Because of his Army service, Spottswood Poles was buried in Arlington National Cemetery.