Re:Reducing Memory Usage in Ruby
by sandGorgon on Wednesday January 24, 2018 @12:10AM
(#16220494)
What I'm really keen to see is truffleruby receiving a lot of community support.
The underlying JVM and the Truffle/Graal/SubstrateVM is the product of millions of man hours of research - and for the very first time, you have a compiler framework on top of that.
Re:Reducing Memory Usage in Ruby
by kawsper on Tuesday January 23, 2018 @07:41PM
(#16219349)
I am so happy that Github have hired Tenderlove to go nuts with these type of projects. His talks are always extremely fun and interesting.
I run Ruby with jemalloc, and I know that samsaffron have tried to include it within the Ruby releases (like Redis does), but progress seems to have been stalled https://bugs.ruby-lang.org/issues/9113
Re:Reducing Memory Usage in Ruby
by johncolanduoni on Tuesday January 23, 2018 @08:27PM
(#16219572)
Kind of a tangent, but does anyone know why glibc’s allocator loses to jemalloc on common workloads? I would expect that would be something worth rectifying unless other requirements tie their hands.
Re:Reducing Memory Usage in Ruby
by __s on Tuesday January 23, 2018 @08:44PM
(#16219667)
Quite a few complaints about jemalloc from Rust, where people wonder about higher than necessary memory usage
jemalloc likely wins more for the kinds of allocations you find in scripting language runtimes, which also happens to be similar to a browser given DOM & JS
Re:Reducing Memory Usage in Ruby
by masklinn on Wednesday January 24, 2018 @03:42AM
(#16221284)
> Kind of a tangent, but does anyone know why glibc’s allocator loses to jemalloc on common workloads?
Jemalloc implements more complex strategies (per-thread caches learned from tcmalloc, preallocated arenas, …).
It tends to have a somewhat higher memory use than simpler allocators, but have faster throughput and less fragmentation (the latter is why Firefox switched to jemalloc by default IIRC).
> I would expect that would be something worth rectifying unless other requirements tie their hands.
Why? If they consider ptmalloc good enough for their purpose, replacing it would be very low priority, and they may value e.g. code simplicity higher.
Re:Reducing Memory Usage in Ruby
by igravious on Tuesday January 23, 2018 @06:43PM
(#16219009)
What I'd like to be able to do is load ActiveRecord (or any other v. heavy lib) asynchronously in non web-based apps so that I can paint the UI quicker. By the time the user will do/need anything db-related a proxy object should be swapped out for an actual ActiveRecord object. Using Rails' ORM is so nice and ergonomic but it kills start-up time with all the file requires, Bootsnap helps a bit but I'd prefer not to have to use it as it is yet one other moving part to consider.
Re:Reducing Memory Usage in Ruby
by AznHisoka on Tuesday January 23, 2018 @09:53PM
(#16219953)
I am actually facing a similar problem. I also find that Rails just creates a db connection everytime I do Model.new even though I have not yet saved it.
Re:Reducing Memory Usage in Ruby
by amichal on Tuesday January 23, 2018 @11:31PM
(#16220329)
This is because active record models learn their attributes by querying the database scheme. I haven’t tried it but there appears to be a way to cache this information in the local file system which would likely avoid the early dB connect if you really needed to do that. Also avoids querying the scheme on processs boot entirely if you are willing to trust the cached scheme matches the. database scheme. See https://github.com/rails/rails/issues/24133 for discussion
Re:Reducing Memory Usage in Ruby
by dragonwriter on Wednesday January 24, 2018 @02:07AM
(#16220930)
> Ruby seems to have a footprint on Hacker News disproportionate to its general popularity and TIOBE index.
TIOBE index is a junk metric (I'm surprised so many technical people seem to buy into it; it's the worst kind of easy to get a precise number, hard to demonstrate that that number has any more than a very distant relationship to anything meaningful, kind of metric that, IME, tech people mock bad managers for buying into.)
That said, Ruby’s got (via Rails and other web frameworks, and via various testing and admin tools) a very strong footprint in use even if it's no longer the hype-of-the-month.
Re:Reducing Memory Usage in Ruby
by lstamour on Wednesday January 24, 2018 @12:19AM
(#16220524)
Rails, and possibly 37signals, was very influential to many folks about a decade ago, especially in the startup community, where Rails was seen as especially useful for prototyping or rapid, test-driven web app development. https://www.quora.com/Why-do-so-many-startups-use-Ruby-on-Ra...
It’s popularity at this may have diminished with the rise of JavaScript, Go and alternatives but many folks still like and use Ruby, or at least would like to see it continue to improve and thrive.
Re:Reducing Memory Usage in Ruby
by berfarah on Wednesday January 24, 2018 @12:15AM
(#16220512)
Ruby has one of the better communities among programming languages - disproportionate to its popularity. I wish more languages had that quality of discourse.
Re:Reducing Memory Usage in Ruby
by abhiyerra on Wednesday January 24, 2018 @01:17AM
(#16220742)
Ruby and Rails have been around for about 10 years at this point. It is stable and for most websites it just works (up to a certain size). It is not as exciting as JS is now but people are using it to get things done since it is relatively mature and used a lot in startups. Likely why you see it referenced a lot here.
Re:Reducing Memory Usage in Ruby
by farnsworthy on Wednesday January 24, 2018 @12:31AM
(#16220559)
From a different perspective, having worked on numerous Ruby projects for years across the private (startups to Fortune 500), nonprofit, and government sectors, spanning many locations with worldwide team members, I don’t see where Ruby/Rails isn't at this point—it’s seemingly all-pervasive.
Similar to general hiring/salary surveys, I’m not sure that the referenced index is accurate.
Re:Reducing Memory Usage in Ruby
by mwaitjmp on Wednesday January 24, 2018 @01:51AM
(#16220871)
Not only due to the popularity of Rails, Ruby has become popular for sysadmins as config management solutions such as puppet and chef are built on top of it. The resulting syntax is very ruby like, and in some cases you are even required to write a few ruby blocks here and there.
It also has an amazing testing framework in the form of rspec, which can be used for much more than just testing Ruby code.
Re:Reducing Memory Usage in Ruby
by flukus on Wednesday January 24, 2018 @02:21AM
(#16220983)
HN is not at all representative of the wider programming community. HN attracts a lot of people working for trendy startup companies and is very silicon valley centric, but that is a tiny portion of the programmers world wide who are much more likely to be working on a java project for a generic business than using the latest fad.
TIOBE has it's issues but as a rough guide to what the industry is actually using I think it's in the ballpark.
Re:Reducing Memory Usage in Ruby
by igravious on Tuesday January 23, 2018 @06:46PM
(#16219021)
Crystal is not a replacement for Ruby. In the same way that many Algol-like languages are not a straight swap for each other. Don't get me wrong, Crystal is a nice idea but say goodbye to all that yummy Ruby meta-programming which is one of the key things what makes the language so appealing in the first place.
Re:Reducing Memory Usage in Ruby
by eropple on Tuesday January 23, 2018 @07:04PM
(#16219127)
Exactly. As a Ruby developer, Crystal makes me itch. It jettisons exactly what Ruby is valuable to me for: quickly and dynamically building fluent-feeling libraries and tools. It's fast, but I don't need fast--if I don't need Ruby's metaprogramming, I have other, better, established languages to use for those tasks. (Kotlin, C#, Rust, whatever.)
Elixir is kind-of in the same boat for me. I think it's neat and has its place, and BEAM is really cool, but it doesn't scratch the Ruby itch for me. I just don't care about it looking vaguely like Ruby if it's not actually gonna be Ruby.
Re:Reducing Memory Usage in Ruby
by sam0x17 on Tuesday January 23, 2018 @08:01PM
(#16219448)
Have you used a recent (2017) version of Crystal? Quickly building fluent-feeling libraries and tools is exactly my use case for Crystal.
For me, everything that Ruby is valuable for is retained. I'm able to do all the things I did in Ruby just fine, and it prevents me from doing a lot of things I shouldn't have been doing.
We recently migrated our app from Rails to Crystal (using the Amber framework) and our code base is now 2/3rds the size and we eliminated hundreds of bugs in the process simply from static typing and explicit nil handling. Our response times are also now all under 30ms where we were at 100ms+ before.
Re:Reducing Memory Usage in Ruby
by eropple on Tuesday January 23, 2018 @08:57PM
(#16219720)
Until I can perform runtime define_method calls and dynamically construct classes, it doesn't matter what Crystal does. It's fundamentally antithetical to my needs. (JavaScript isn't, but JavaScript is kind of clunky.)
I have Kotlin, C#, C++, and Rust if I need static typing. Ruby is for when I don't.
Re:Reducing Memory Usage in Ruby
by sam0x17 on Tuesday January 23, 2018 @10:30PM
(#16220090)
People think "oh, it's compiled, so I can't have dynamic methods" but that couldn't be more far from the truth. You can do all of that at compile time, and Crystal provides an excellent way of doing so. E.g.: https://github.com/sam0x17/crystal-mongo-orm/blob/master/src...
Re:Reducing Memory Usage in Ruby
by yorwba on Tuesday January 23, 2018 @09:13PM
(#16219784)
How late at runtime do these dynamic definitions need to be possible? Just at startup or also after the program has been running for a while?
I occasionally wonder whether it might be possible to accommodate more dynamic language features with a compiler that treats them like a macro system that is expanded statically, such that no actual dynamicism is necessary at runtime.
Re:Reducing Memory Usage in Ruby
by notalex on Wednesday January 24, 2018 @10:02AM
(#16223175)
Reading well written source code creates a more efficient workflow than reading api docs. For me, the fact that Crystal is written in Crystal gives it a far greater advantage.
Plus it is faster than Go/Java/Elixir. As others have pointed out, Crystal does have meta programming capabilities.
Re:Reducing Memory Usage in Ruby
by eropple on Tuesday January 23, 2018 @10:37PM
(#16220111)
You can dynamically build at runtime an HTTP client from an OpenAPI spec (unavailable until runtime, to be clear), method-by-method, using macros? Tell me more.
Re:Reducing Memory Usage in Ruby
by look_lookatme on Wednesday January 24, 2018 @07:57AM
(#16222313)
Yeah use Ruby. I prefer to have well defined interfaces for the objects I use. I love Ruby to death, but it's frustrating to encounter code like this:
[:thing, :other_thing].each do |msg|
define_method(msg) do
@json[msg].to_s
end
end
When this is far more clear and, for me, aesthetically pleasing:
def thing
@json[:thing].to_s
end
def other_thing
@json[:other_thing].to_s
end
I'm not sure how you'd use a ruby object that maps to an OpenAPI spec if you don't know what's in the API response until runtime... surely you or the user of your library has an idea what's in the API they are consuming?
Re:Reducing Memory Usage in Ruby
by eropple on Wednesday January 24, 2018 @12:17PM
(#16224343)
You may have an idea, sure, but the alternative is to codegen a new instance of the client library any time anything changes rather than keeping up to date with the API as it evolves and exists. (The moral equivalent is pulling a WSDL during app startup and making sure that you're presenting a correct interface to it.)
I full-stop don't do codegen in any project under any circumstance that is not done strictly within the workflow of a build tool (i.e., a Maven generated-sources gizmo). OpenAPI's options are pretty much all exactly not that: all the codegen options's happy paths seem to be vendoring generated code. And that makes me itch, too.
Re:Reducing Memory Usage in Ruby
by yxhuvud on Wednesday January 24, 2018 @11:22AM
(#16223896)
The example in question is perfectly possible to do in Crystal during compile-time, using macros. What isn't possible is to create a new class on the fly as a sort of magic lookup table that magic methods is called upon. Good riddance, for that functionality.
Re:Reducing Memory Usage in Ruby
by eropple on Wednesday January 24, 2018 @01:52AM
(#16220876)
Sure. Or I get effectively the same thing with JavaScript. (I just don't much like Node--it's not bad but it's not great--and the dev experience for specifying something like a DSL is a bummer.)
Re:Reducing Memory Usage in Ruby
by eropple on Wednesday January 24, 2018 @12:42AM
(#16220599)
If you're doing stuff with OpenAPI (as I've been doing over the last few weeks), it becomes a pressing thing. =( Generated clients usually are pretty awful, you usually have to generate your own (because they don't publish gems when their APIs change, which is why I've written online-generated ones!), and once you've completed the bootstrap process for an HTTP client, it's as fast as any other Ruby thing. Which is not the fastest out there by a long shot--though JRuby helps--but is Fast Enough for most stuff.
I'm also not sure Crystal can implement an active record pattern off of a database? (Though I am not personally the type to go with that, I prefer row mappers.)
Re:Reducing Memory Usage in Ruby
by eropple on Wednesday January 24, 2018 @12:18PM
(#16224351)
"You have to specify your fields at compile time" is exactly not like ActiveRecord, though. That's exactly the downside to it for doing ActiveRecord.
At that point, you may (should) just use an object mapper like ROM that has no model-class hooks at all. (Which I actually like. But the Rails world disagrees.)
Re:Reducing Memory Usage in Ruby
by sam0x17 on Monday January 29, 2018 @04:10PM
(#16260642)
You effectively do specify your fields before runtime in Rails, though, in the form of your migrations. Furthermore all of the associations (belongs_to, etc), are arguably also specified before runtime since those helpers run when the class is parsed.
Re:Reducing Memory Usage in Ruby
by eropple on Friday February 09, 2018 @02:20AM
(#16338521)
I frequently don't use Ruby-based migrations. I use datastores defined elsewhere and use Sequel as a convenient and easy interface for tasks against those datastores.
You're trying to slice too fine. Crystal can't do this. That's OK. Own it, don't try to hide it.
Re:Reducing Memory Usage in Ruby
by semperdark on Wednesday January 24, 2018 @03:24AM
(#16221208)
Wow. I've been checking in on the Crystal ecosystem every few weeks, but I somehow missed Amber. It feels like someone has finally cracked the code that made Rails great.
Re:Reducing Memory Usage in Ruby
by sam0x17 on Tuesday January 23, 2018 @10:36PM
(#16220108)
Take a look at macros in crystal and I promise you 99% of the time you will be able to do what you did in Ruby. Practically everything people do that is "necessarily dynamic" could be done with macros at compile-time. You CAN do things like define the behavior for when a method name is unmatched on an Object or (at compile time) do some string processing and define a bunch of methods based on the results.
The underlying JVM and the Truffle/Graal/SubstrateVM is the product of millions of man hours of research - and for the very first time, you have a compiler framework on top of that.
The Truffle framework also brings in a lot of support that all languages get for free, like zero overhead profiling https://twitter.com/nirvdrum/status/948333404122214401
Truffleruby is already in the running in benchmarks, and it will be awesome to see how far it can be pushed . http://nirvdrum.com/2017/02/15/truffleruby-on-the-substrate-...
It seems right now, TruffleRuby 99% of Ruby code without much problem. C Extension may be the key to wide spread adoption.
[1] https://github.com/graalvm/sulong
I run Ruby with jemalloc, and I know that samsaffron have tried to include it within the Ruby releases (like Redis does), but progress seems to have been stalled https://bugs.ruby-lang.org/issues/9113
jemalloc likely wins more for the kinds of allocations you find in scripting language runtimes, which also happens to be similar to a browser given DOM & JS
See also: https://sourceware.org/glibc/wiki/389-ds-malloc
Jemalloc implements more complex strategies (per-thread caches learned from tcmalloc, preallocated arenas, …).
It tends to have a somewhat higher memory use than simpler allocators, but have faster throughput and less fragmentation (the latter is why Firefox switched to jemalloc by default IIRC).
> I would expect that would be something worth rectifying unless other requirements tie their hands.
Why? If they consider ptmalloc good enough for their purpose, replacing it would be very low priority, and they may value e.g. code simplicity higher.
Incidentally, there was talk at one point to replace glibc's ptmalloc: https://sourceware.org/ml/libc-alpha/2014-10/msg00419.html
TIOBE index is a junk metric (I'm surprised so many technical people seem to buy into it; it's the worst kind of easy to get a precise number, hard to demonstrate that that number has any more than a very distant relationship to anything meaningful, kind of metric that, IME, tech people mock bad managers for buying into.)
That said, Ruby’s got (via Rails and other web frameworks, and via various testing and admin tools) a very strong footprint in use even if it's no longer the hype-of-the-month.
It’s popularity at this may have diminished with the rise of JavaScript, Go and alternatives but many folks still like and use Ruby, or at least would like to see it continue to improve and thrive.
Similar to general hiring/salary surveys, I’m not sure that the referenced index is accurate.
It also has an amazing testing framework in the form of rspec, which can be used for much more than just testing Ruby code.
TIOBE has it's issues but as a rough guide to what the industry is actually using I think it's in the ballpark.
Elixir is kind-of in the same boat for me. I think it's neat and has its place, and BEAM is really cool, but it doesn't scratch the Ruby itch for me. I just don't care about it looking vaguely like Ruby if it's not actually gonna be Ruby.
For me, everything that Ruby is valuable for is retained. I'm able to do all the things I did in Ruby just fine, and it prevents me from doing a lot of things I shouldn't have been doing.
We recently migrated our app from Rails to Crystal (using the Amber framework) and our code base is now 2/3rds the size and we eliminated hundreds of bugs in the process simply from static typing and explicit nil handling. Our response times are also now all under 30ms where we were at 100ms+ before.
I have Kotlin, C#, C++, and Rust if I need static typing. Ruby is for when I don't.
I occasionally wonder whether it might be possible to accommodate more dynamic language features with a compiler that treats them like a macro system that is expanded statically, such that no actual dynamicism is necessary at runtime.
Plus it is faster than Go/Java/Elixir. As others have pointed out, Crystal does have meta programming capabilities.
I full-stop don't do codegen in any project under any circumstance that is not done strictly within the workflow of a build tool (i.e., a Maven generated-sources gizmo). OpenAPI's options are pretty much all exactly not that: all the codegen options's happy paths seem to be vendoring generated code. And that makes me itch, too.
You can build your code at runtime from the spec and then compile it at runtime to get pretty good performance.
I'm also not sure Crystal can implement an active record pattern off of a database? (Though I am not personally the type to go with that, I prefer row mappers.)
https://github.com/amberframework/granite-orm https://github.com/sam0x17/crystal-mongo-orm
At that point, you may (should) just use an object mapper like ROM that has no model-class hooks at all. (Which I actually like. But the Rails world disagrees.)
You're trying to slice too fine. Crystal can't do this. That's OK. Own it, don't try to hide it.
However, that is basically a lookup table using string based dispatch to decide what to execute, and such a lookup table can be done in anything.
Ruby is awesome, as is Elixir, but very different awesome.