<< >>
justin = { main feed , music , code , askjf , pubkey };
[ <<< last (older) article | view in index | next (newer) article >>> ]

July 8, 2019
macOS screen updating, part III: 2019

Five years ago, in the year of our lord 2014, I wrote about the difficulties of drawing bitmapped graphics to screen in macOS, and I revisited that issue again in the winter of 2017.

Now, I bring you what is hopefully the final installment (posted here for usefulness to the internet-at-large).

To recap: drawing bitmapped graphics to screen was relatively fast in macOS 10.6 using the obvious APIs (CoreGraphics/Quartz/etc), and when drawing to non-retina, normal displays in newer macOS versions. Drawing bitmapped graphics to (the now dominating) Retina displays, however, got slow. In the case of a Retina Macbook Pro, it was a little slow. The 5k iMacs display updates are excruciatingly slow when using the classic APIs (due to high pixel count, and expensive conversion to 30-bit colors which these displays support).

The first thing I looked at was the wantsLayer attribute of NSView:

  • If you use "mynsview.wantsLayer = YES", things get better. On normal displays, slightly, on a Retina Macbook Pro's display, quite a bit better. On a 5k iMac's display, maybe slightly better. Not much.
  • Using the wantsLayer attribute seems to be supported on 10.6-current.
  • For views that use layers, you can no longer [NSView lockFocus] the view and draw into it out of a paint cycle (which makes sense), which prevents us from implementing GetDC()/ReleaseDC() emulation.

After seeing that enabling layers wasn't going to help the 5k iMacs (the ones that needed help the most!), I looked into Metal, which is supported on 10.11+ (provided you have sufficient GPU, which it turns out not all macs that 10.11 supports do). After a few hours of cutting and pasting example code in different combinations and making a huge mess, I did manage to get it to work. I made a very hackish build of the LICE test app, and had some people (who actually have 5k iMacs) test the performance, to see if would improve things.

It did (substantially), so it was followed by a longer process of polishing the mess of a turd into something usable, which is not interesting, though I should note:
  • If you want to update the entire view every time, you can configure a CAMetalLayer properly and just shove your bits into the CAMetalLayer's texture and tell it to present and avoid having to create another texture and a render pipeline and all of that nonsense.
  • If you want to do partial window updates (partial-invalidates, or a GetDC()-like draw), then you have to create a whole render pipeline, a texture, compile shaders, blah blah blah ugh.
  • There's a bunch more work that needs to get done to make it all work (and adapt to changing GPUs, blah blah blah)...
This stuff is now in the "metal" branch of swell in WDL, and will go to master when it makes sense. This is what is in the latest +dev REAPER builds, and it will end up in 6.0. I'm waiting for it to completely bite me in the ass (little things do keep coming up, hopefully they will be minor).

As one final note, I'd just like to admonish Apple for not doing a reasonable implementation of all of this inside CoreGraphics. The fact that you can't update the 5k iMac's screen via traditional APIs at an even-halfway-decent rate is really stupid.

P.S. It does seem if you want to have your application support Dark Mode, you can't use [NSView lockFocus] anymore either, so if you wish to draw-out-of-context, you'll have to use Metal...






6 Comments:
Posted by Daniel X on Sun 04 Aug 2019 at 10:38 from 174.56.59.x
I was having the same problem (and have been following your saga since 2017, when I first became aware of similar issues, first with Cairo, now with my own library)

Anyway, I just stumbled upon somebody's git commit and it solved my issue (and how!)

github.com/aseprite/laf/commit/cf9...

My code which draws a CGImage, then some more stuff on top of it, went from < 5fps full screen to 100+, on a 5k iMac.


Posted by Justin on Mon 05 Aug 2019 at 20:40 from 108.30.215.x
ah nice! I should try the drawsAsynchronously attribute... though I might still need to use metal in order to do partial updates and out-of-context updates...


Posted by Chris on Fri 20 Sep 2019 at 10:13 from 204.13.44.x
Wow, wantsLayer and drawsAsync made a night and day difference here - thanks for this deep dive. The obscure, ever changing nature of this issue has definitely been a bear.


Posted by Colin on Wed 27 Nov 2019 at 04:09 from 93.220.15.x
Thanks for looking into this. Much appreciated!


Posted by Justin on Sun 01 Dec 2019 at 10:13 from 108.30.215.x
Just a note -- if you use wantsLayer/drawsAsync, you need to make sure your source bitmap is persistent and not resized until it gets drawn. So it's a bit tricky sometimes.


Posted by Justin on Sun 01 Dec 2019 at 10:46 from 108.30.215.x
and by tricky I mean, not practical for real world use (you can *hope* that the system will draw your bitmap before you need to resize it in response to a user action, but can you ever know???).


Add comment:
Name:
Human?: (no or yes, patented anti crap stuff here)
Comment:
search : rss : recent comments : Copyright © 2019 Justin Frankel