> but there is no way they are going to overcome the ecosystem deficiencies to make it worth while in any unbiased cost benefit calculation
Is there a concrete reason for that? A lot of the approach in Elixir is to not go the route of other languages and merely slap an adapter interface on Python libraries. The approach is to approach things from a very Elixir and BEAM specific point of view. This is how Livebook was developed and how the machine learning libraries are being developed.
In fact, one could view it as Python being the one that's going to struggle to overcome the limitations set forth by its own language design. Elixir comes with very tightly integrated tooling (projects, testing, type analysis, static analysis, notebooks, scripting, etc.) that basically has 100% adoption rate without a plethora of third-party competitors and lack of integration with the language such as Python has. With immutability and concurrency built into the language, Elixir has a leg up, and there is active research into bringing static typing to Elixir and also binding Elixir to faster runtime environments, such as Rust.
Python simply cannot make the jump to Elixir, Erlang, and BEAM's language and VM capabilities like they can to a machine learning ecosystem akin to Python's.
People are willing to put in thousands if not millions of working hours into shoehorning things into the language they already know, unwilling to learn a language that already has a good set of basic principles and concepts, that would enable the thing they envision.
People will build huge C and C++ libraries for realizing distributed systems that they can then use from Python, instead of getting to know a language on the Beam VM, because they only want to stay in the mainstream ecosystems or are unwilling to deal with new concepts like in Elixir or Erlang (for starters lets say lightweight processes, distributed system, pattern matching, proper recursion (TCO), the functional programming view of things. Some of those are very scary for many Python developers.).
Fortunately there are those on the other side as well, stubborn enough to try to implement things in those more niche languages that they prefer, based on technological merit of basic principles and concepts of a language, rather than just doing what everyone else does. So sometimes something really great and interesting comes out of that.
Often however, the shoehorning masses are simply too many, the people knowing the other ecosystems too few, to keep up with them or the newest developments, further enabling the "but this is what everyone else is doing!" kind of mindset. Something something "beating the averages" here.
Maybe, but you make it sound like it's always some Blub paradox inspired reason. That someone doesn't want to learn thing <X> might be they're totally fine with <Y>; they can get work with it, they like working with it, whatever.
Not everyone is always looking for the next thing, better or not.
Static analysis in Elixir is quite more accessible because you rely way less frequently on dynamic dispatch than Python (or Ruby or JS). You typically know which module you are calling to, structs and pattern matching tell you about fields (which we verify at compile-time) and primitive type information. Things like deprecated and undefined functions are part of the compiler, while most other dynamic languages typically require type systems or linters (or do it exclusively at runtime).
I agree on type analysis though. It is better than nothing but that's not much to talk about. We are working on it. :)
Ok fair, compared to other dynamically typed languages Elixir’s static analysis is better, by virtue of existing at all :-) Something is better than nothing, but getting “function has no local return” when you make a particular kind of typo three modules down the stack is a long way from eg Java or TypeScript just telling you about the typo, instantly, right where you made it. That’s the norm IMO and that’s what I was referring to.
I’m excited to hear that Elixir may be getting a bit closer to that, and I agree with your observation that because typical Elixir code uses dynamic dispatch very little, there’s a lot of room for improvement in ways that eg Ruby will never be able to offer.
I’m sorry about the “terribly, laughably” bit of my comment.
For what is worth, I was not considering Dialyzer as part of my reply. You should be getting many warnings related to typos from the compiler!
Anyway I agree with the sentiment: we have a high ceiling but we are currently far away from it. If we had a Dialyzer that runs all the time, is fast (Erlang/OTP 26 already improved here), and has good error messages, it would already be great.
We are currently researching a proper type system into the language. Fully integrating and improving the Dialyzer experience would be our plan B.
Wow that would be great! I recall reading that an effort some years ago to do a type system on top of Erlang failed because typechecking inter process communication got too messy (or slow). Personally, I feel like a type system that leaves message passing completely out of scope would solve so many problems already. And then people can invent typechecked GenServers and the likes in userland.
I mean when 2 NodeJS processes communicate via JSON over HTTP then those aren’t typechecked either, even if their codebases are all TypeScript. Except, of course, when using the same TS rpc lib on both sides of an interface, with some code sharing between them so they use the same types.
I feel like even when all typechecking happens only on regular static synchronous function calls, and message passing is left out of scope, it’ll be so extremely useful. And like you said, the low amounts of dynamic dispatch makes it tractable. Still a huge amount of work I bet though :-)
Worse than Python? It's a question; I don't know. I do know that the two most terrible (for me; I like types and a working IDE that helps me, not fight me) languages (js/py) are also the most popular. I don't know what that means, but as I did build large systems with Erlang in the past (in banking) which are still running production with massive load (and don't fall down in 2 weeks because the ENTIRE ecosystem had a minor version update, invalidating all the ABI's), I cannot imagine it's worse than Python or JS.
Python has at least a couple of functional type checking systems. Elixir only has Dialyxer, which has never actually worked for me any time I’ve attempted to use it. (It’s not just me. At work we have a medium-sized Phoenix codebase and about every fifth person who joins is determined to add Dialyxer to our CI pipeline - so far without success.)
I use Dialyzer on all side projects and it always works. However, I enforce via Credo that everything gets a typespec, and I employ type-driven design.
Dialyzer and Credo for Phoenix does require some workarounds, as that project isn't interested in providing typespecs or base compliance with Credo and is very heavy on macros.
I believe you. However, the current Dialyzer/Dialyxir experience bad enough that the creator of Elixir is working on an effort to specify an entirely different† gradual typing system for the language:
This strongly suggests that Dialyxir is not working as effectively for most users as e.g. Typescript or Mypy.
† I can imagine people quibbling with 'entirely different', but the proposed type system does have a new(ish) theoretical basis that's still the subject of active research.
By type analysis, I mean Dialyzer and built-in type errors, and by static analysis, I mean Credo and its ability to define custom checks and Elixir's ability to access and walk its own AST plus the compiler features.
Yes, Dialyzer could be better, but it's pretty good if you write typespecs. But Credo is quite good and one of the best such tools I have ever used. And Elixir g enerally has excellent error messages. Why do you say it's laughably bad?
Also, the point was primarily that all these tools are effectively built-in to the language without several different competing libraries.
And I neglected to mention the official formatter.
My last weekend Project in Elixir is roughly 3 yrs ago now, but from what I recall the system was leagues better then pythons optional typing. It's just bad if you're used to languages that have been built around types from the start.
I think you’re just saying that you prefer dynamically typed languages. The point here is that Python now has much better options than Elixir if you want to check types statically.
I'm not sure how you could've gotten that idea from my comment. If I put it on a line, I'd say (no typing) < (python optional types) < (elixirs structs and pattern matching) < (typescripts types|interfaces) < (real static types from Java etc)
Hmm, ok, but structs and pattern matching aren’t static types. They certainly don’t offer anything like the kind of type safety you can obtain (with some effort) with mypy or other Python type systems. Mypy is quite similar to Typescript.
Is there a concrete reason for that? A lot of the approach in Elixir is to not go the route of other languages and merely slap an adapter interface on Python libraries. The approach is to approach things from a very Elixir and BEAM specific point of view. This is how Livebook was developed and how the machine learning libraries are being developed.
In fact, one could view it as Python being the one that's going to struggle to overcome the limitations set forth by its own language design. Elixir comes with very tightly integrated tooling (projects, testing, type analysis, static analysis, notebooks, scripting, etc.) that basically has 100% adoption rate without a plethora of third-party competitors and lack of integration with the language such as Python has. With immutability and concurrency built into the language, Elixir has a leg up, and there is active research into bringing static typing to Elixir and also binding Elixir to faster runtime environments, such as Rust.
Python simply cannot make the jump to Elixir, Erlang, and BEAM's language and VM capabilities like they can to a machine learning ecosystem akin to Python's.