using new for each node and value, combined with virtual dispatch tends to be a c++ anti-pattern. It looks like you are writing other languages in C++ syntax, motivated by promises of speed.
The actual benefits of C++ come when you approach problems differently. This is a case where more exposure to C helps you avoid all the Java isms.
Things to consider:
- can you allocate memory for the whole system?
- can you make types homogenous so they can fit in tight arrays (unions are common for nodes)
- can you batch similar types
- specially for auto diff/math can you represent operations as a stack instead of a tree?
I am only bringing this up because you said your goal was to learn C++.
> - can you allocate memory for the whole system? - can you make types homogenous so they can fit in tight arrays (unions are common for nodes) - can you batch similar types - specially for auto diff/math can you represent operations as a stack instead of a tree?
these are good questions, thank you! I'm "learning" c++ in a completely different way - looking at books rather than looking at existing code. Appreciate this comment!
Some of these questions I had thought about, but was learning more towards "describe the function in some higher level representation and then 'compile' it down to something computable and autodiff-able". This is exactly where my mind goes when I think about allocating memory for the whole system. that felt more like a racket/lisp way of looking at the problem.
- GUI frameworks work with an order of 10 or 100 long lived widgets. Number crunchers work on the order of millions of values that exist for a few cycles
- would anybody recommend you should reach for C++ to write Java style OOP in 2024?
using new for each node and value, combined with virtual dispatch tends to be a c++ anti-pattern. It looks like you are writing other languages in C++ syntax, motivated by promises of speed.
The actual benefits of C++ come when you approach problems differently. This is a case where more exposure to C helps you avoid all the Java isms.
Things to consider:
- can you allocate memory for the whole system? - can you make types homogenous so they can fit in tight arrays (unions are common for nodes) - can you batch similar types - specially for auto diff/math can you represent operations as a stack instead of a tree?
I am only bringing this up because you said your goal was to learn C++.
> - can you allocate memory for the whole system? - can you make types homogenous so they can fit in tight arrays (unions are common for nodes) - can you batch similar types - specially for auto diff/math can you represent operations as a stack instead of a tree?
these are good questions, thank you! I'm "learning" c++ in a completely different way - looking at books rather than looking at existing code. Appreciate this comment!
Some of these questions I had thought about, but was learning more towards "describe the function in some higher level representation and then 'compile' it down to something computable and autodiff-able". This is exactly where my mind goes when I think about allocating memory for the whole system. that felt more like a racket/lisp way of looking at the problem.
The Java isms were C++ best practices in C++ GUI frameworks before Java was even an idea, there is even a famous book about them.
- GUI frameworks work with an order of 10 or 100 long lived widgets. Number crunchers work on the order of millions of values that exist for a few cycles
- would anybody recommend you should reach for C++ to write Java style OOP in 2024?
- Best practice according to who?
First of all, it is C++ OOP style, Java cloned C++, not the other way around.
Secondly, all major surviving C++ GUI frameworks are still using the same style.
Best practice according to well known folks in the computing industry, with more impact than any of us will ever have on our lifetimes.
I think the main point here is that while this style might make sense for a GUI framework it's not so great for a numerical library like this.
For example: my library is really slow.
And instead of blaming OOP, have you actually used a profiler?
Are you suggesting that many individual allocations and virtual dispatch are not impactful, or just advocating use of a profiler?
Once again, the context of this thread is someone taking an interest in C++ to write fast autodiff code.
The actual C++/CUDA code is here:
https://gitlab.com/mebassett/quixotic-learning/-/tree/master...
about 1,000 LoC overall.