The OS won’t matter much in the beginning, though it helps that you’re already using Linux as you likely already have Python and GCC installed.

I don’t think you need a better PC than what you already have if the only goal is to learn programming, so I’d spend that money on something else.

I’d suggest you go through Harvard’s CS50 if you’ve never been exposed to computer science before: https://www.harvardonline.harvard.edu/course/cs50-introduction-computer-science . You can audit it for free, you don’t really need to pay for the certificate (which IMO doesn’t have much value at that level anyway).

Also, try to get into a computer science degree if you want to do that as a career, bootcamps and MOOCs are nice additions but will never replace a real degree.

Roughly one year. No GPU code however for that project as the target library is CPU-only anyway so not really comparable to PyTorch (and PyTorch is more than just the autodiff), but there was lots of SIMD vectorization. Yeah you could train a neural network on CPU with it if you want, and the expression template stuff I talked about would be somewhat equivalent to PyTorch’s operator fusion, but the target use is more quant finance code.

Automatic differentiation in C++17, had to do it from scratch as they weren’t open to 3rd-party libraries and it had to integrate with many symbolic calculations that are also done in the library (so even if you used a 3rd-party library you’d still have to rewrite parts of it to deal with the symbolic stuff). Succeeded in doing it and basically the performance was on par with what people in the industry would expect (all derivatives at once in around 3 times the evaluation time, sometimes much less if the calculation code has no dynamic parts and is differentiated entirely at compile-time).

It was pretty cool because it was a fun opportunity to really abuse template meta-programming and especially expression templates (you’re basically sort of building expression trees at compile-time), compile-time lazy evaluation, static polymorphism and lots of SFINAE dark magic and play around with custom memory allocators.

Then you get scolded by the CI guys because your template nightmare makes the build times 3x slower, so the project then also becomes an occasion to try out a bunch of tricks to speed up compilation.

Have developers be more mindful of the e-waste they’re contributing to by indirectly deprecating CPUs when they skip over portions of their code and say “nah it isn’t worth it to optimize that thing + everyone today should have a X cores CPU/Y GB of RAM anyway”. Complacency like that is what leads software that is as simple in functionality as equivalent software was one or two decades ago to be 10 times more demanding today.

I don’t see the point of the video. The whole thing is just him looking at a screen.

Linear and logistic regression are much easier (and less error prone) to implement from scratch than neural network training with backpropagation.

That way you can still follow the progression I suggested: implement those regressions by hand using numpy -> compare against (and appreciate) sklearn -> implement SVMs by hand using cvxpy -> appreciate sklearn again.

If you get the hang of “classical” ML, then deep learning becomes easy as it’s still machine learning, just with more complicated models and no closed-form solutions.

I’d say since you’re a beginner, it’s much better to try to implement your regression functions and any necessary helper functions (train/test split etc…) yourself in the beginning. Learn the necessary linear algebra and quadratic programming and try to implement linear regression, logistic regression and SVMs using only numpy and cvxpy.

Once you get the hang of it, you can jump straight into sklearn and be confident that you understand sort of what those “blackboxes” really do and that will also help you a lot with troubleshooting.

For neural networks and deep learning, pytorch is imposing itself as an industry standard right now. Look up “adjoint automatic differentiation” (“backpropagation” doesn’t do it any justice as pytorch instead implements a very general dynamic AAD) and you’ll understand the “magic” behind the gradients that pytorch gives you. Karpathy’s YouTube tutorials are really good to get an intro to AAD/autodiff in the context of deep learning.

Reimplementing stuff from scratch, overengineering and, if you’re coding in a compiled language, knowing a bit of assembly to be able to make better performance-related decisions.

EDIT to clarify the overengineering part: the idea is not to do it at work obviously because there you will have to meet deadlines and you will have to spend time on what’s most important, but the idea instead is to do that when working on your own personal projects. Try to make every component of your project as “perfect” as possible as if you were trying to rebuild a Mercedes-AMG car. That’s how you’ll learn a bunch of new tricks.

Wrote it in an awkward way but yeah I meant to say the section where you can find the corresponding entry 😬

Yup that definitely does the same thing.

If anyone else is wondering why the 3 is there, it’s because usually you won’t find just one printf. You have the printf user command, the printf function from the standard C library, and POSIX manual entries for both the printf user command and C function. The id number is then an identifier for the corresponding section of the printf entry, and you can list all of them by doing a man -f printf.

Yup! Try also man malloc 😁

If you want multi-line code, you need to put it like this:

For these kinds of questions, your best friend is the documentation. In particular, a man 'printf(3)' yields:

Format of the format string

The format string is a character string, beginning and ending in its initial shift state, if any. The format string is composed of zero or more directives: ordinary characters (not %), which are copied unchanged to the output stream; and conversion specifications, each of which results in fetching zero or more subsequent arguments. Each conversion specification is introduced by the character %, and ends with a conversion specifier. In between there may be (in this order) zero or more flags, an optional minimum field width, an optional precision and an optional length modifier.

The overall syntax of a conversion specification is:

%[$][flags][width][.precision][length modifier]conversion

This. I might also add that joining an acting class can help a lot with social skills.

They also claimed to be faster than C++ in the beginning, but they failed to mention that the C++ code was unoptimized single-thread code while the Mojo version was SIMD-vectorized and multithreaded. It’s only after people criticized them for that that they started calling the C++ code “scalar C++”. Scientific rigor is lacking in their benchmarks.

I mean I know they promised to do it, but it’s sad that they still haven’t open-sourced the toolchain. The naming is awkward though, I’d have loved if they called it Python++ instead as it’s supposed to be a superset of Python and would have made it easier to present to your boss.