No harder than it is to sympathize with people who say that any of the judges is an extremist. In 2010 alone, 7 of the judges were the lone dissenter at least once:

Breyer - Milner v. Department of Navy Alito - Snyder v. Phelps Sotomayor - United States v. Jicarilla Apache Nation Ginsburg - Kentucky v. King Thomas - Pepper v. United States Kennedy - Global-Tech Appliances, Inc. v. SEB S. A. Scalia - Ransom v. FIA Card Services, N. A.

Roberts and Kagan are the only justices I haven't found an example of where they were lone dissenters.