News for hackers. Stuff that matters.
Re:Proposal of a new concurrency model for Ruby 3 [pdf] by catnaroek on Thursday September 08, 2016 @10:31AM (#12453325)
> A guild can't access the objects of other guilds.

> 2. Transfer membership or Move in short

How is this enforced? What exactly happens at runtime if a guild tries to manipulate an object that belongs to another? (Absent a compile-time check, this is always a possibility.)

Re:Proposal of a new concurrency model for Ruby 3 [pdf] by dragonwriter on Thursday September 08, 2016 @02:31PM (#12455792)
> How is this enforced? What exactly happens at runtime if a guild tries to manipulate an object that belongs to another? (Absent a compile-time check, this is always a possibility.)

It would seem that:

(1) Guild ownership would have to be tracked in the runtime, obviously.

(2) Any access from Ruby code in the runtime, the runtime would also know what Guild the access request came from as well as the Guild the object belonged to against which access was sought.

(3) The runtime would be required to fail in some well-defined way (presuming, raising an exception in the requester) when the rules were violated.

It should be reasonably straightforward to assure this for all accesses within the runtime, since you can just make sure that there is no method to request access which isn't always attached to the Guild that the request comes from. It may be possible to break the runtime with poorly-behaved extension code that subverts the normal mechanisms, and it may be impossible to fully protect against that, but that's pretty much always a potential with extension code.

Re:Proposal of a new concurrency model for Ruby 3 [pdf] by catnaroek on Thursday September 08, 2016 @02:33PM (#12455828)
How would you transfer ownership of big linked data structures? See my other comment: https://news.ycombinator.com/item?id=12455566

I'm not particularly worried about C extensions. I already know those are a lost case.

Re:Proposal of a new concurrency model for Ruby 3 [pdf] by dragonwriter on Thursday September 08, 2016 @02:47PM (#12456005)
> How would you transfer ownership of big linked data structures?

Very carefully?

More seriously, I think with guilds, what you absolutely don't want to do is build yourself into a position where you ever want to move a big linked data structure (that is, unless you know you are only going to use it in one guild, you never want to build a big linked data structure of mutable objects.)

Big structures of mutable objects should be guild local (or external, in a store that has its own controls for concurrent access.)

Re:Proposal of a new concurrency model for Ruby 3 [pdf] by _ko1 on Thursday September 08, 2016 @05:58PM (#12457727)
absolutely.
Re:Proposal of a new concurrency model for Ruby 3 [pdf] by pmontra on Thursday September 08, 2016 @01:51PM (#12455330)
It's similar to Rust's transfer of ownership. Rust is compiled so you get a compile time error if you try to access something you can't access anymore.

Check the code at http://rustbyexample.com/scope/move.html and run it (inside the page). There is a commented out println towards the end. Comment it in, run the code again and see the compiler error.

More about transfer of ownership at https://doc.rust-lang.org/book/ownership.html

Re:Proposal of a new concurrency model for Ruby 3 [pdf] by catnaroek on Thursday September 08, 2016 @02:13PM (#12455566)
I already know very well how Rust works. What I'm not convinced of is that ownership can be correctly and efficiently enforced purely by runtime mechanisms. (Presumably Ruby's implementors aren't interested in introducing static checks anytime soon, right?)

For example, how would you transfer ownership of a big linked data structure from one guild to another?

(0) In Rust, this is as easy as handing ownership of the root node (an O(1) operation) to another thread. Ownership is transitive: whoever owns the root node also owns whatever the root node owns.

(1) In Ruby, I don't see how transitive ownership could possibly work. If I understand the proposal correctly, every object is owned directly by a guild, never by a parent object. Thus, you would have to traverse the entire linked data structure to transfer ownership of every node. This is O(n) work. Making things worse, you would have to make sure that the ownership transfer is atomic - no other part of the program should see the data structure in a “partially transferred” state.

Another example: Say you initially have a guild with three objects, Foo, Bar and Qux, where Foo and Bar point to Qux. If I transfer ownership of Foo, should Qux be transferred as well?

(0) In Rust, the type system forces me to explicitly distinguish between the following possibilities:

(0.a) Foo exclusively owns Qux, and Bar merely borrows it. In this case, Foo and Qux are frozen, and thus can't be transferred until Bar's borrow ends.

(0.b) Bar exclusively owns Qux, and Foo merely borrows it. In this case, transferring Foo doesn't change the fact Qux is owned by Bar.

(0.c) Foo and Bar jointly own Qux (using an Arc). In this case, transferring Foo doesn't change the fact Qux is jointly owned.

(1) In Ruby, what exactly should happen here? Have these guys really thought about the possibilities?

Re:Proposal of a new concurrency model for Ruby 3 [pdf] by _ko1 on Thursday September 08, 2016 @05:57PM (#12457722)
> Thus, you would have to traverse the entire linked data structure to transfer ownership of every node. This is O(n) work.

Correct.

> Making things worse, you would have to make sure that the ownership transfer is atomic - no other part of the program should see the data structure in a “partially transferred” state.

Correct. Guild::Channel#transfer_ownership() does it.

Basically, share big linked data with multiple threads is difficult (simply we need to lock every access).

> Another example: Say you initially have a guild with three objects, Foo, Bar and Qux, where Foo and Bar point to Qux. If I transfer ownership of Foo, should Qux be transferred as well?

Foo -> Qux; Bar -> Qux

Yes, Qux also moved. Programmer can know by "exception" when accessing Qux via Bar after transfer.

This "realization" is the key of Guild. On threads, we can't realize that Qux is shared mutable.

Re:Proposal of a new concurrency model for Ruby 3 [pdf] by catnaroek on Thursday September 08, 2016 @07:33PM (#12458406)
Thanks. This clarifies a lot of things.
Re:Proposal of a new concurrency model for Ruby 3 [pdf] by parenthephobia on Thursday September 08, 2016 @06:37PM (#12458033)
> For example, how would you transfer ownership of a big linked data structure from one guild to another?

Slowly.

Objects well-suited to transfer would be shallow mutable structures with immutable data "underneath". Immutable data can be shared, so they'd be ignored by the transfer logic.

Another approach likely to be common would be not transferring objects at all, but sending proxy objects to other guilds which transparently marshal method invocations between those guilds and the object's home guild. For large mutable complexes which are intertwined with everything a guild does, that's probably a more manageable approach.

> Making things worse, you would have to make sure that the ownership transfer is atomic - no other part of the program should see the data structure in a “partially transferred” state

This is easy. You just run the transfer operation with the guild's mutex locked. No other guild can have a reference to a mutable object, and an immutable object doesn't need to be transferred.

> In Ruby, what exactly should happen here? Have these guys really thought about the possibilities?

It seems obvious that every linked mutable object is invalidated, so Qux will be transferred.

Re:Proposal of a new concurrency model for Ruby 3 [pdf] by catnaroek on Thursday September 08, 2016 @07:33PM (#12458404)
> Objects well-suited to transfer would be shallow mutable structures with immutable data "underneath". Immutable data can be shared, so they'd be ignored by the transfer logic.

How often are data structures designed like this in Ruby?

> Another approach likely to be common would be not transferring objects at all, but sending proxy objects to other guilds which transparently marshal method invocations between those guilds and the object's home guild.

What if said “home guild” ends up overburdened with requests coming from all over the place?

> This is easy. You just run the transfer operation with the guild's mutex locked.

You mean both the sender and the receiver's mutexes? I'm worried about the receiver being able to observe partial transfers.

Re:Proposal of a new concurrency model for Ruby 3 [pdf] by eregon on Thursday September 15, 2016 @08:05AM (#12505353)
The object graph would only be handed to the receiver once it's fully transferred I would imagine.
Re:Proposal of a new concurrency model for Ruby 3 [pdf] by riffraff on Thursday September 08, 2016 @11:00AM (#12453610)
you get a runtime error
Re:Proposal of a new concurrency model for Ruby 3 [pdf] by catnaroek on Thursday September 08, 2016 @01:32PM (#12455132)
Is it a guaranteed runtime error, or a best-effort thing, à la Java's ConcurrentModificationException?
Re:Proposal of a new concurrency model for Ruby 3 [pdf] by riffraff on Thursday September 08, 2016 @01:51PM (#12455333)
as I understand it, a guaranteed error, it's at page 50 of the slide

    • Accessing from the source guild is invalidated
    • Cause exceptions and so on
    • ex) obj = “foo”
          ch.transfer_membership(obj)
          obj.upcase #=> Error!!
          p(obj) #=> Error!
Re:Proposal of a new concurrency model for Ruby 3 [pdf] by _ko1 on Thursday September 08, 2016 @05:59PM (#12457732)
Yes.