The main problem with Java (or garbage collected languages in general) as a first language is needing to unlearn the bad habits it ingrains when you move to a systems programming language with manual memory management. Other than that it's a pretty good first language, though I'd suggest learning a bit of C at the same time just to get a basic grip on things like pointers and stack vs heap.

Edit: it occurs to me that C# would be the perfect learning language. It's very similar to Java and an easy first language, but you'd also learn about stack allocation through structs, and can teach pointers using unsafe (though I think unsafe code is still GCed, so this wouldn't help with the memory management side of things. Haven't touched C# in fifteen years so I'm not sure how it works anymore).