Sparks - Latest Comments

Re: Coroutines 5: Performance of C++03, C++20 and Python generators

Michael Sparks — Fri, 18 Jul 2025 19:15:08 -0000

Regarding the C++20 version, as mentioned in all the posts, the code (+Makefile) for all the posts is here:

* https://github.com/sparkslabs/blog-extras

The specific branch for testing with -O3 is in this branch :
* https://github.com/sparkslabs/blog-extras/tree/o3_coroutine_tests

And as mentioned in my comment, the delta between the two is here:

* https://github.com/sparkslabs/blog-extras/compare/main...o3_coroutine_tests

Bear in mind the delta is in 5-bench-cpp03coroutine.cpp

As for not doing the 22 iterations (or not) in the C++03 version, feel free to take a look.

However, I've done some cursory checks by creating a new branch and updating the benchmarking version ...

https://github.com/sparkslabs/blog-extras/blob/o3_coroutine_tests_rolling/by-date/2023/Coroutines/0/5-bench-cpp03coroutine.cpp

... to check intemediate values, inject the number of iterations from outside, and keep the generated intermediate assembly (-fverbose-asm -save-temps -v), and keep count of the number of values created. It's butt-ugly, but it does seem to force the values to be generated. (However, I don't think it's preserved the original intended approach - which makes the benchmark moot)

You can see the specific changes here:

* https://github.com/sparkslabs/blog-extras/commit/e2ae338e231986add69e4bb489808d5737edaa45

That's triggered by a new target: make bench-cpp03coroutine.s

For convenience I pass that through nl (line numbering). I've added the result of that to the branch as well - which you can see here:

https://github.com/sparkslabs/blog-extras/blob/o3_coroutine_tests_rolling/by-date/2023/Coroutines/0/nl-bench-cpp03coroutine-5-bench-cpp03coroutine.s

As you can see from the cpp file, this throws in a switch statement to check intermediate values (which actually sped things up), and then incrementing a counter based on that (which appears to have prevented optimisation - which makes sense), and then eventually injecting the number of loop iterations.

The final result there is 35s on the wall clock, with 23,000,000,000 values generated - which is about 1.5ns per value, unless I'm losing it.

Also, but bear in mind the code now has extra overhead of checking a variable iterlimit rather than a fixed value of 22, has to do an extra work. (the switch statement will itself obvious be optimised so I'd assume this would still be a low overhead) then a comparison for the correct running value and finally increment a counter.

... and still I think after the optimisation flags are enabled it ceases being a useful benchmark of coroutine overheads - just a benchmark of how well the optimiser works, since the resulting code differs significantly from the sort of code I'd use these sort of generators for in python. (eg -- https://github.com/sparkslabs/guild/blob/master/examples/blog/miniguild/media.py )

ie I tend to use these sorts of things for managing concurrency, not for performance - since it tends to make the code clearer IMO.

Also, FWIW, this sort of thing is the reason why I didn't use any optimisation flags in my benchmarking, because I didn't want to have to guess if my code wasn't be changed beneath me . (Which I knew any self respecting optimiser would :-) ) After all, the optimiser has no way of knowing what I'm trying to measure and just tries to keep the calculations correct, whereas I was more interested in benchmarking the how.

After all I wasn't after "how fast does this go" - I was after "how fast is 'this' relative to 'that' and to 'the other' and for that you want to ensure that you don't optimise out the thing you want to test.

After all, as I said, I wanted to get an idea of the naive difference between C++20 and python, and only looked at my ancient C++03 based code from over 20 years ago as an afterthought. The fact it outperformed the C++20 code by such a wide margin with no optimisations enabled surprised me, which led to more investigation.

Anyhow, glad you've found this interesting - the only place I've used this particular C++03 iteration form is in my python to C++ compiler to replace one of the standard python iterators range() with direct equivalent in the generated C++ code. The reason that uses C++03 form rather than C++20 coroutines was because that code was written around 10 years ago - predating C++20 coroutines. (So again, the point there, was readability, not performance)

All that said, I think, if anything, this conversation has led me right back to the original conclusion:

* Python is obviously and unsurprisingly slowest.
* C++20 is second slowest for this sort of thing, somewhat surprisingly.
* Sometimes hidiously naive C++03 code can work out surprising well :-) (However, I'd only transform code to that form if I really needed the performance that would come with it, because that form wouldn't be obvious to most people and more awkward to debug. )

Re: Coroutines 5: Performance of C++03, C++20 and Python generators

Michel — Thu, 17 Jul 2025 13:43:26 -0000

I see that you found the bug in the C++03. I suspected as much and had replaced the macros with actual code and had initialized __generator_state to 0 and noticed it gave me the correct answer after that. I guess the optimizer saw it could be any value, so why not initialize it with a value that would make it skip some code! :D
Can you show me your C++20 version? I am getting way much faster result than you. I will have to push your new code to godbolt to see what the generated assembly does. I suspect it is not doing the 22 iterations of the loop because 0.9ns/iteration is too fast to do what has to be done. I suspect it may unroll the inner loop fully and simplify the operations, even pre-computing the result.

Re: Coroutines 5: Performance of C++03, C++20 and Python generators

Michael Sparks — Thu, 17 Jul 2025 07:21:46 -0000

Hi Michel,

Thanks for the feedback.

FWIW, when I wrote this set of notes a couple of years ago, I was mainly curious about how C++20 coroutines hang together in comparision to python ones. The comparison to C++03 was an afterthought initially and then something I thought worth capturing in the blog when the results presented turned out. Indeed, I'd been playing with C++20 coroutines on my break on and off for a couple of weeks before thinking about adding them. TBH, I didn't expect the C++03 coroutines to still work when I first tried them.

I deliberately didn't switch on optimisations when doing this because I didn't want to see how well the compiler would throw everything away :-) (I realise it'll do some trivial things even with -o0 but that was deliberately picked to switch almost everything off)

I found it interesting that under o3 optimisation the C++03 operates differently. Since the occurs with -O1, -O2, -Os as well, but not -O0, I assumed it was to do with initialisation of the generator, based on it using the trick of using __LINE__. Forcing that to be initialised to zero fixes that issue - so there was a latent bug there.

A quick search shows that HALO optimisation only happens with -o3

Also, you're right though - g++ versions do matter. The version I was using back in 2023, was the version that shipped with ubuntu 22.04.02LTS. The support then wasn't great, and people were talking about C++23, but it wasn't fixed at that point.

These days I'm using ubuntu 24.04.02LTS, and the g++ version that uses is 13.3.0

Re-running both benchmarks with -O3 in that version results in C++03 version exitting almost immediately, so as expected that optimised out all the code AFAICT. (Though interestingly performs correctly when actually used, so that probably just needs a minor tweak)

Rerunning the C++20 version with -03 results in that finishing in 136.6s on my laptop vs 1531.6s with -O0 - which is quite the hefty increase in performance, and nice to see. (-Os doesn't get that result) (Just a touch over 6ns/iteration) However, see below...

I think the easiest way to fix the problem with the -O3 build of the C++03 version and to ensure that the C++20 version isn't cheating as well might be to simply take the final value from the generator, and check that it's correct and increment a counter. (Thought that could still be gamed)

.. so I've just tried this - cf: https://github.com/sparkslabs/blog-extras/compare/main...o3_coroutine_tests

The C++03 version, with the minor initialisation correction, and the check on final returned value now takes 20s overall on wall clock on my machine -- about 0.9ns/iteration. (this is compared with 253s wallclock for -O0)

The C++20 version with -03 and the check on final returned value ? 338s. (vs 1531s wallclock for -O0 on current machine)

So, I think the key result is still true:

* Python is still slowest - which isn't unexpected at all.
* C++20 on -O3 is almost as good as C++03 without optimisation, which is useful to know since that's "avoids a rewrite" territory.
* However for C++03 and C++20 with -O3 ? My old, naive, relatively weird C++03 code beats C++20 by over a factor 16 which surprises me even more than before. (Caveat, I'd hope C++23 stdlib generators to perform better)

I don't view these results as particularly scientific BTW, just interesting ballparks. (I'd want to really run these on a machine which isn't running any other loads/etc if I cared to do the benchmarks "properly" :) )

Again, thanks for the feedback, hope you found it interesting :-)

Re: Coroutines 5: Performance of C++03, C++20 and Python generators

Michel — Wed, 16 Jul 2025 13:58:15 -0000

You should try your benchmarks again. With g++15 with full optimization, on my machine, the C++20 version takes 1 minute and 3s.
Be careful that because you do not use any of the computed numbers, the optimizer will reduce your code to an empty one. It can also understand what the inner loop computes and shunts it out. You have to change the loop limit so that the compiler cannot cheat.
There is however something very strange with the C++03 version when the optimization is turned on. The Fibonnaci generator yields 1, 2, 3, 5,... missing the first 1. It may be a compiler bug.

PS: I suspect that when you did your test, the compiler did not do HALO optimization for the coroutine frame. Now it does and 3ns/iteration is definitely fast.

Re: Restarting Blogging

David Boddie — Sun, 20 Jun 2021 15:57:54 -0000

Good to see you blogging again!

Re: Escaping The Panopticon of Things?

Michael Sparks — Mon, 12 Feb 2018 04:30:29 -0000

Note: a whistle stop example of how to use IOTKit to build device oriented and user centric IOT devices is in the next post on this blog - which you can find here:

* http://www.sparkslabs.com/m...

Re: Serving packages for multiple distribitions from a single PPA

the way of the future — Thu, 18 Aug 2016 10:51:21 -0000

please please please please please make Pixie HUGE for Arduinos! Down with C++ !

Re: Hello Microbit

Michael Sparks — Fri, 12 Feb 2016 07:20:38 -0000

Thank you for you kind words :-)

A few months later I was able to post a blog about the prototyping process here:

http://www.bbc.co.uk/rd/blo...

And then last september gave a talk about the process in more detail:

https://www.youtube.com/wat...

That said, seeing all the happy faces on twitter, and seeing the overwhelmingly positive response has been humbling to see :-) Really looking forward to seeing the children receiving their devices, and looking forward to see what they do with it :-)

Once again though, thank you for your kind words :-)

Re: Hello Microbit

Pete Bell — Fri, 12 Feb 2016 02:48:10 -0000

Thank you, Michael! I love them, my students are chomping at the bit (!) for them and I think you are being modest about the impact these will have. Fantastic job, sir!

Re: Sparks

John Regan — Mon, 05 Oct 2015 14:12:30 -0000

I had A BBC Micro in 1983-85 16KB that got upgraded to 32KB and used to code on that in BASIC. I was into it then but after the save to tape feature failed and never received it back from repair (shop just shut down with all contents gone!) I sort of lost interest because I could not afford a new one. Even then I was sad because I had all sorts of ideas of how a program could , turn on the lights, let me know when someone was at the front door, easy stuff really with some added hardware but I did think that it could by voice activation without going to the home PC. I am sure a lot of kids at the time had the same kind of idea especially with doctor who and other sci fi TV series. I am still sad though because I never forget the frustration of a program not running and the joy of finding out why. I had to make money at the end of the day since I left school and went to become an electrician then Joined the armed forces. These days I am stuck with cyber threat security removal but still not able to program a computer. I just wish I got my BBC Micro back because even the ideas I have today are still not thought of! Well I don't think they are I don't see the idea around anyway. I even see that people can make apps but I do not have the first idea of making one. I miss it all.
John

Re: Sparks

Michael Sparks — Tue, 09 Jun 2015 06:33:05 -0000

Updated link for original paper -- http://www.serena.com/docs/...

Re: Ubuntu PPA Packages of sparkslabs projects

Michael Sparks — Sun, 01 Mar 2015 12:49:00 -0000

NB Sparkslabs just means the projects I have on github under the id "sparkslabs"

Re: Guild - pipelinable actors with late binding

Michael Sparks — Wed, 12 Mar 2014 17:24:53 -0000

If I was to implement copying, it wouldn't be hidden. (though it might be wrapped up in Actor that does it for you)

Re: Guild - pipelinable actors with late binding

Jim Easterbrook — Wed, 12 Mar 2014 09:28:08 -0000

Please, no (hidden) copying. As well as the performance overhead there's a problem copying things like SWIG-wrapped C++ objects.
It also needs to be remembered that the sender usually retains a reference to the sent object. This is not often a problem but could trip up a naive user who decides to use socket.recv_into() with a buffer that was sent out earlier.
I think this is down to self discipline when writing Kamaelia or Guild components. Any attempt to enforce it would get in the way, I think.

Re: Guild - pipelinable actors with late binding

Michael Sparks — Tue, 11 Mar 2014 05:39:50 -0000

Hi,

Regarding one to many - yes that's not a bad convention, or perhaps it'd be goodwould wrap duplicated messages in a wrapper that warn the recipient that they have a duplicated reference. Also, while it's not mentioned above, I have implemented the backplane component from kamaelia, along with a bridging mode. That could be modified to enforce copying rather than sharing, or to hint to recipients that they have received a shared reference. It's tempting to think about having some kind of copy on write mechanism/dictionary.

Assuming the Camera, Display, and Framestore component from above, then you could do this: (untested, but gives you the idea - there's an example of the backplane in the examples directory)

from guild.components import *

actors = camera, framestore, display = Camera(), Framestore(), Display( (800,600) )
Backplane('FRAMES').go()
pub = PublishTo('FRAMES')
sub1 = SubscribeTo('FRAMES')
sub2 = SubscribeTo('FRAMES')

pipeline(camera, pub)
pipeline(sub1, framestore)
pipeline(sub2, display)

start(*actors)
wait_KeyboardInterrupt()
stop(*actors)
wait_for(*actors)

Perhaps worth noting that this is one area where a little bit better syntactic sugar would be useful :-)

Thanks for spotting the typo - I'll fix that! (Typoes here are generally because I'm writing this on public transport :-)

Re: Guild - pipelinable actors with late binding

Jim Easterbrook — Tue, 11 Mar 2014 04:43:33 -0000

Very interesting, Michael.

It reminds me of Qt's signal/slot mechanism, which in recent versions of PyQt is quite user friendly. It also uses decorators to mark 'input' methods as slots. Sending a message to a slot (as the user sees it) directly invokes the slot method, if sender and receiver are in the same Qt thread. When I start playing with Guild a PyQt interface is probably one of the first things I'll try.

What are your thoughts on one-to-many connections? (E.g. bind the camera output to inputs of two separate displays.) If all receiving methods are obliged to treat input objects as immutable this should be safe, shouldn't it?

PS Spotted one typo - "small/just big though" should be "small/just big enough"?

Re: Sparks

Anthony Kyne — Tue, 06 Aug 2013 08:24:09 -0000

Good read, thanks.

I started coding in the 80's on a BBC Micro for a few reasons.

1) My interest in robots, maths and puzzles and wanting to make my computer talk like the car in Knight Rider. Which with the Welcome program was pretty easy. But them implementing that into other programs

2) All my friends had Spectrums or Commodore 64's with cool games that I weren't available on the BBC. So I learn't how to program so I could make games like that.

3) It was actually possible to make games of the standard that you were buying in the shops as a 10 year old working on a machine in your parents dining room. I knew everything was possible.

I'm lucky that I work in the games industry and been able to make cutting edge games for the last 17 years, but if I was working on my own I wouldn't have been able to make anything marketable in that period and I think that's the big difference now. Point 3 above meant that I could make Paperboy by myself but now-a-days I can't make Heavy Rain for example.

Reading this article though has put a thought in my head. If the BBC got together with Unity and some big names in the UK games industry (Ian Livingstone, Peter Molyneux, etc) where over a period of weeks on a TV show a game would be developed within the Unity engine with free downloadable assets, created by the BBC from the Unity Asset Store. People could see what and how to make a fun, great looking game while learning a great language like C#

They can also mess around with the code and assets to create a more personalised version of the game

Re: Sparks

Tim Urban Crap — Tue, 26 Mar 2013 11:05:37 -0000

It seems people are catching up with this notion. I've heard lots of discussions recently (esp. on Today programme) about how we lost out by not teaching code in schools. A lot of it tied up with economics and the fear that China will be all-powerful. So back in the headlines because of a perceived loss of revenues somewhere?? Missing the point by a mile, I feel. Kids won't learn code because the Gov fear a loss of national economic dominance. They would learn it because it is its own reward.

Learning code is something akin to learning to paint and draw (as in your quote), you can become a professional, or you can use it purely for your own fulfilment. Kids today, when they feel there's something worth knowing, go online, find what they want to learn and teach themselves. And anecdotally, I showed the clip (pasted below) about learning this stuff to my two nephews (age 8 and 10) and both are now learning Python and HTML5 - essentially teaching themselves. Looks like technology will look after us if the curriculum will not.

http://techcrunch.com/2013/...

Re: Hack to the Future (H2DF), Preston, 11/2/12

Michael Sparks — Sun, 19 Feb 2012 19:13:40 -0000

For people who worry about such things, this in no way states BBC opinion on anything. They're my opinions. If they were BBC opinions, they'd be on the BBC website, and probably not have my name attached. :-)

Re: Sparks

npinkard — Sat, 24 Dec 2011 07:49:42 -0000

Michael

I am reading this from the States after you posted to the Hackersaurus group. You are dead on in your analysis of the importance of understanding the constraints of the time we live in and designing around them. We need to analyze the ecosystem of the past and learn from it and not necessarily repeat it. We are in a period where students don't feel that they can create the games they interact with on the computer and/or game console. What is interesting is that students do feel they can create the movies they watch and the music they listen to which leads them to spend time creating in these spaces. I think a major reason for this belief is the existence of multiple channels for discovering and sharing videos and songs created by non-professionals (e.g. youtube, soundcloud, tumblr, CDs (old school music sharing), mp3 players). These channels easily allow users to interact with their peers around the videos and songs they are creating.

Currently, we don't have a "sexy" enough universal way to share user created games which means that the ability to create games isn't viewed as a vehicle for increasing social capital (or gaining cool points). Most kids spend time doing things that they believe is in reach and can be shared with their peer group.

We can look at the Scratch in some ecosystems as an example of what is possible. In some school communities where Scratch has been introduced it is viewed as a valuable skill set within the kids' peer groups which results in kids spending hours at home creating games and sharing via the Scratch website. Unfortunately, scratch games are not available yet on the iPad/iphone which means that in order for a kids to share they need to be sitting around a computer screen. Also, as students get older, Scratch looses some of its engagement. I think this is partly because of the inability of most kids to create graphically appealing sprites and the lack of easy access to sprite libraries that are age and interest appropriate.

I think that if we can enable kids to easily share their games created with systems like Gamestar Mechanic, Scratch, and others on their current gaming consoles, tablets, phones or create a universally available and portable console that kids find desirable (the gaming version of the Leap tools) then we may be able to create the conditions that will engage kids in creating, playing and sharing games and other interactive objects as valuable social capital.

Best,

Nichole

P.S. Happy Holidays

Re: Sparks

Michael Sparks — Mon, 19 Dec 2011 07:30:43 -0000

Maybe - it's either something like that or like minds :-) It's a great rationale behind hack to the future though. I've now signed up btw.

Re: Sparks

Alan O'Donohoe — Mon, 19 Dec 2011 05:02:40 -0000

Michael. I have read this blog post a couple of times already before today. I start to wonder now if I subconciously took the idea for Hack To The Future http://h2df.eventbrite.co.uk from your line "After all, the world of 30 years ago looks as old and archaic to the children of today as 1955 did to Marty McFly."

In recent months, some developer/programmer types have said to me words to the effect of "I wish when I was at school somebody had shown me what was possible with computer science" - that's my rationale behind Hack To The Future - allowing children to meet their potential future selves to discover the power of computing science.

Re: Sparks

Michael Sparks — Mon, 19 Dec 2011 04:59:12 -0000

Follow up blog was delayed. Plans have been changed by discovery of http://www.playmycode.com/ whilst investigating tools to build something very similar, almost identical to it! It's great :-)

Re: Sparks

Michael Sparks — Wed, 19 Oct 2011 04:41:56 -0000

I think it's worth noting that I recognise that despite the national curriculum/GCSEs etc coding and computer studies etc didn't drop out from schools completely. However for the large part it has, and realising "it was never really quite there the way you thought" does matter. Also, I phrased the title as a question because I don't claim to have the definitive answer, the above is really just my best guess based on the facts I knew then and know now.

Re: Sparks

Michael Sparks — Mon, 17 Oct 2011 19:31:05 -0000

I think in a way that's my point. Many of us who learnt to code in the 80's did so on machines that weren't BBC Micros.
However, I think without it legitimising micros in the eyes of parents, the home micro market may wellt've been as large, as early, nor as targetted at kids. In a way, this song sums it up pretty well http://www.youtube.com/watc... ("Hey Hey 16K - We bought it to help with your homework")