• 2 Posts
  • 31 Comments
Joined 1 year ago
cake
Cake day: June 28th, 2023

help-circle







  • My advice comes from being a developer, and tech lead, who has brought a lot of code from scientists to production.

    The best path for a company is often: do not use the code the scientist wrote and instead have a different team rewrite the system for production. I’ve seen plenty of projects fail, hard, because some scientist thought their research code is production level. There is a large gap between research code and production. Anybody who claims otherwise is naive.

    This is entirely fine! Even better than attempting to build production quality code from the start. Really! Research is solving a decision problem. That answer is important; less so the code.

    However, science is science. Being able to reproduce the results the research produced is essential. So there is the standard requirement of documenting the procedure used (which includes the code!) sufficiently to be reproduced. The best part is the reproduction not only confirms the science but produces a production system at the same time! Awws yea. Science!

    I’ve seen several projects fail when scientists attempt to be production developers without proper training and skills. This is bad for the team, product, and company.

    (Tho typically those “scientists” fail to at building reproducible systems. So are they actually scientists? I’ve encountered plenty of phds in name only. )

    So, what are your goals? To build production systems? Then those skills will have to be learned. That likely includes OO. Version control. Structural and behavioral patterns.

    Not necessary to learn if that isn’t your goal! Just keep in mind that if a resilient production system is the goal, well, research code is like the first pancake in a batch. Verify, taste, but don’t serve it to customers.




  • I use Scala. Huge fan. Both for work and personal projects. Not wildly popular, so i get asked this by my team. This is what i tell them:

    Focus on the patterns, design and theory. Those last longer than the language. What is enabled by language? What is difficult? What does the type system let you prove? How do you utilize the type system? How does the execution of the language work? What is the runtime like?

    Language development itself has evolved. There is no longer a huge jump from one language to another because, well, we’ve figured out a lot of it. Want to learn rust? The patterns and concepts you learn with Scala will still apply. Go? Same, just a different form and you’ll probably be asking about monads in short order. :)

    If i look at future trends. Real far stuff. Stuff that will only become popukar years from now. Well, some of it exists now in esoteric languages. Those languages wont be useful for jobs, but studying them now preps you for the future.

    Elixir is further future than Go. Go, like Java was, is not particularly novel but a very solid implementation of what we currently know.







  • I’ve experienced similar.

    One thing that helped: separating research code from production.

    Research is to answer a decision problem and much of the optimization problems that are in software engineering do not, and should not, apply.

    Once the research problem is answered. Reproducing that answer with production quality systems should be it’s own project. This also serves as a reproduction of the science in the research. Satisfying that hallmark of the scientific method.

    Course, getting a company to agree to such an arrangement is near impossible. Especially if they have never been crippled by the mismatch expectations of putting research code in production.

    As that is an organizational problem not just an engineering problem, good luck convincing management.

    An alternative, if I can’t get such an arrangement is the building a platform that supports integration of research code. That can be… uh… hard. Aside from the people challenges not all tech can support such a platform.



  • What… What problems does Nix solve? throws down his beer What value is precision? Why make a cube about 10cm per side when you can make a cube 10.001 cm ± 0.001 cm? Do you want software that’s a collection of found parts that just happens to work? Or a system engineered to precise requirements?

    Rant aside: that sums one difference. Both containers and Nix solve an encapsulation problem. They solve them differently. Containers gives software their own namespace. Nix requires software to exist in an a universal namespace. “/bin/bash” may be different between containers. While “/nix/store/bash-82828def8282829whatever/bin/bash” is always the exact same bash in Nix.

    Precision has a cost but sometimes the precision is necessary. Eg: nix is great building closures that contain exactly the software requested and no more. While containers are more imprecise: take a base and add on additional stuff. From a software supply line perspective this can be exactly the precision required.

    Nixpkgs is (afaik) the closest thing to Amazon’s internal package system. So the issues it solves is definitely valuable… To at least Amazon scale orgs.

    As a dev who likes to tweak their system Nix offers an unparalleled ability to alter deep dependencies and correctly propagate those through everything. Wanna alter libc and rebuild everything - jvm and all - for some Java service? Yep. Nix will handle the build no problem.

    Excessive? Sometimes - plenty of systems work fine when dependencies are mutated underneath. However, when there is a need there is NixOS in a class of it’s own.

    Also, they are complementary solutions: nix is great at building containers.