Yeah. And most reports on LTE include "poor battery life". That will likely change eventually, but..
It's already changing. The initial issue was that the phones stuffed two discreet cellular modem chips in. The LTE chips were first generation, meant more for MiFi and USB WAN adaptors. The mobile phone market has become so competitive, phone makers used these first gen chips.
More recent LTE phones offer a single chip solution using second, or now even third generation LTE modems. The Nexus 4 has
one of these inside it, but the LTE portion is disabled.
Rumors seem to be that Google I/O may see a slight bump in the Nexus 4, enabling CDMA and LTE, along with offering larger flash storage. Seems reasonable, since the work to do this is minor and could fit in well to a midcycle update to the phone.