Show HN: Magnetissimo – Self-hosted torrent search engine written in Elixir

Re:Show HN: Magnetissimo – Self-hosted torrent search engine... by sergiotapia on Saturday January 28, 2017 @04:15AM (#13505580)

Hope you guys like this and use it! If it's hard to use, tell me. My goal is to make this simple to run anywhere. Part of that will be to use Distillery to create compiled executables.

I rewrote this after an initial version that used Redis as a queue to process. Asking end users to install Redis was a step too far I think, so I stepped back and thought about how to solve this without external dependencies.

I ended up with a simple Elixir and Erlang queue implementation. It works much simpler, just as fast, and no freaky deps. GenServer starts up a worker for each crawler, and it schedules work by itself. It's really strange to use a language so complete, it kind of feels like cheating. Programming in easy-mode.

Pull requests very welcome to the crawler folder, it's super easy to write a crawler for your favorite site!

https://github.com/sergiotapia/magnetissimo/tree/master/lib/...

[ Parent | on HN ]

Re:Show HN: Magnetissimo – Self-hosted torrent search engine... by gyrgtyn on Saturday January 28, 2017 @02:45PM (#13508538)

you could make it a nerves project too, maybe after you get rid of some of those database dependencies.

It'll make you a ~100Mb linux distro that will boot straight into your app in ~10 seconds on a raspberri pi.

http://nerves-project.org/

[ Parent | on HN ]

Re:Show HN: Magnetissimo – Self-hosted torrent search engine... by sauvage255 on Saturday January 28, 2017 @02:57PM (#13508679)

very useful project, thanks for sharing this!

[ Parent | on HN ]

Re:Show HN: Magnetissimo – Self-hosted torrent search engine... by memla on Saturday January 28, 2017 @10:26AM (#13506671)

If the goal is to make it simple to run, why not support SQLite?

[ Parent | on HN ]

Re:Show HN: Magnetissimo – Self-hosted torrent search engine... by brightball on Saturday January 28, 2017 @10:32AM (#13506703)

Seems like the goal is avoiding dependencies and he's got 3 built in database choices for that already. DETS or Mnesia would probably work well for this.

[ Parent | on HN ]

Re:Show HN: Magnetissimo – Self-hosted torrent search engine... by sergiotapia on Saturday January 28, 2017 @01:34PM (#13507914)

Thank you, I'll take a look at what those are. If there's a way to remove Postgres from the equation, I'm all for it.

[ Parent | on HN ]

Re:Show HN: Magnetissimo – Self-hosted torrent search engine... by yellowapple on Saturday January 28, 2017 @01:45PM (#13507970)

Mnesia is probably what you want in that case, since it turns (D)ETS into a full-fledged database rather than a simple key-value store (think SQL v. Redis). Both are part of Erlang/OTP, though, so you get them for free just by using Erlang/Elixir/LFE/whatever.

There's a hex.pm package called "Amnesia" that wraps Mnesia in a more Elixiry format.

[ Parent | on HN ]

Re:Show HN: Magnetissimo – Self-hosted torrent search engine... by sergiotapia on Saturday January 28, 2017 @02:06PM (#13508109)

That sounds fantastic. I'm all for trimming the fat if we don't need it. I'll research this, thanks for the suggestion.

[ Parent | on HN ]

Re:Show HN: Magnetissimo – Self-hosted torrent search engine... by elcritch on Saturday January 28, 2017 @05:52PM (#13510022)

If you're using Phoenix or Ecto, look into ecto_mnesia (https://github.com/Nebo15/ecto_mnesia). It wraps up mnesia into a nice ecto wrapper. It's limited on what your primary keys can be.

[ Parent | on HN ]

Re:Show HN: Magnetissimo – Self-hosted torrent search engine... by macintux on Saturday January 28, 2017 @01:45PM (#13507969)

DETS (and ETS, the in-memory equivalent) are quite simple for basic K/V-ish storage needs. If you need a relational database, Mnesia should do the trick.

[ Parent | on HN ]