thicket_hdmi

thicket_lightning

One of these things is not like the others. Images courtesy Josh Ott.

Apple’s iOS has at times shown tremendous promise as a portable canvas for art and visual performance for projection. But the “upgrade” to the Lightning port has brought an unfortunate step backward in fidelity – enough of one to make you hang onto an iPad 3 for now, or simply switch back to laptops.

And while the image output from the iPad has gotten murkier, at least the reason why is getting clearer. The good news: Lightning itself seems isn’t to blame, and a fix seems both possible and likely.

We covered the iPad 2′s output capabilities with real enthusiasm back in 2011 – to the point of including some code for developers:
iPad 2 Goes Native: 1920×1200 Output with Thicket; Devs, How to Add This To Your Apps

Scroll down through comments, and you’ll see something has gone a bit … awry since the release of the iPad 4.

Joshue Ott, who wrote the story on HDMI out on iPad for CDM, is the developer of a beautiful iOS audiovisual app called Thicket. (See our interview and profile.) Thicket’s crisp, minimalist 3D geometries mean that it’s a canary in the coal mine when it comes to any kind of compression artefacts. In December, I got a worried email from Josh.

So far, it looks video out using the new lightning/HDMI connector is not direct video out … On iPhone5, thicket output looks absolutely awful. The lines are barely visible as lines, everything is smudged and bad. on iPad mini (I borrowed from a friend) the image is greatly improved, and even usable, but still compressed (harder to tell here but fast movements make it obvious).

Josh sends along video, too, in case the images here weren’t scary enough. The difference is stunning, though to see it properly, you’ll want to click through to Vimeo and download the original video.

remember compare 2 from superdraw on Vimeo.

The results are disturbing. You still get reasonably flexible output options from the iPad and iPhone, and generally speaking, Apple’s devices are capable of some beautiful visuals. But the output adapters suffer when showing signals that are susceptible to overly-aggressive compression. And the new adapters are unmistakably a step backward, not forward, from what came before.

Mostly, we want to see this fixed – and without knowing the cause, it’s hard to know whether or not that can happen.

It came as little surprise when Panic Software, long-time Mac developer (Audion forever!), weighed in with similar results. It was a bit surprising what they found when they took the Lightning AV adapter apart:
The Lightning AV Adapter Surprise [Panic Blog]

Confused by what they saw, they speculated the adapter was some sort of embedded AirPlay gadget. That doesn’t make any sense, so I was relieved when an anonymous commenter, seeming to come from (cough) inside Apple, explained.

Airplay is not involved in the operation of this adapter.
… The reason why this adapter exists is because Lightning is simply not capable of streaming a “raw” HDMI signal across the cable. Lightning is a serial bus. There is no clever wire multiplexing involved. Contrary to the opinions presented in this thread, we didn’t do this to screw the customer. We did this to specifically shift the complexity of the “adapter” bit into the adapter itself, leaving the host hardware free of any concerns in regards to what was hanging off the other end of the Lightning cable.
… It’s vastly the same thing with the HDMI adapter. Lightning doesn’t have anything to do with HDMI at all. Again, it’s just a high speed serial interface. Airplay uses a bunch of hardware h264 encoding technology that we’ve already got access to, so what happens here is that we use the same hardware to encode an output stream on the fly and fire it down the Lightning cable straight into the ARM SoC the guys at Panic discovered.

Certain people are aware that the quality could be better and others are working on it. For the time being, the quality was deemed to be suitably acceptable. Given the dynamic nature of the system (and the fact that the firmware is stored in RAM rather then ROM), updates **will** be made available as a part of future iOS updates. When this will happen I can’t say for anonymous reasons, but these concerns haven’t gone unnoticed.

[Emphasis mine]

An anonymous comment (ahem)

So, in other words, the Lightning adapter is piggy-backing on the output stream that Airplay uses – and the problem is that the Airplay encoding isn’t as good as what we’ve gotten used to. That makes the most sense, and seems the logical conclusion from what Panic saw, more than the one they drew.

I hope, actually, that these images help illustrate that the quality is not “suitably acceptable.”

But there’s a reason to be optimistic:
1. The encoding is happening on the device, and can be updated in the OS.
2. The quote above seems to suggest Apple is aware of the issue and working on it.

More images, which should illustrate why this is important:

thicket_iphone5

thicket_lightning2

My sense is that this will get fixed (and the Web talking about it can’t hurt, either). If it does, of course, we’ll be the first to let you know.

Until then, it could all make you long for these more innocent times:

Hang onto that Dock Connector iPad.

And, yes, laptops still have their advantages, if fidelity is your priority.

  • The Beep

    So funny. I did capture a lot of stuff at work today from some i-devices. We decided on the old dock connector and an iPad3. At random. Just because it felt right having a big plug. Glad we did that now.

  • acampbellpayne

    makes me glad that i’m saving up for a refurb’d ipad 3.

  • http://twitter.com/natcl Nathanaël

    What I’m wondering is if the Lightning connector has enough bandwith to support sending uncompressed frames… If they are compressing it before going in the cable as it was stated I doubt there’s an easy fix…

    • http://pkirn.com/ Peter Kirn

      I guess the question is, if it’s a serial connection, how fast is the serial connection between the iPad and the dongle?

  • Ingo Italic

    Peter, i think you can run a whole series on how the introduction of lighting dock made things worse for iOS musicians and visualists. For example: i have to use two adapters to be able to play my USB Midi gear with the latest iPad 4 – a lightning to 30 adapter + the older camera connection kit. At least it still works.. it seems things changed inside Apple since Scott Forstall left the building.

    • http://pkirn.com/ Peter Kirn

      This decision would have had to be made while Forstall was still there. In fact, I would imagine the original plan for this happened under Jobs. But yeah, the Lightning Adapter is the latest in the size trumps function – I’d say nothing has changed since, oh, the late 90s. I’m just hoping they make up what we lost.

  • Amaury

    No Lightning-enabled device to test here, but I would love to know about latency using the new converter.

  • Chuck

    It can’t be fixed. And wont be fixed. Lightning doesn’t have the bandwidth to send out lossless 1080p 30fps output. Terrible move by Apple – but as they said; there’s only 0.01% of users who use the HDMI output. So to them, they don’t care – doesn’t effect their revenue. Most other vendors have gone with micro USB which has the same problems.

  • bridd

    Does anyone know if this is still an issue post iOS7 ?

  • Macmerlin

    Does IOS 8 improve anything with this HDMI adapter lag?