talipozturk

co-founder of Vectroid: We forked Lucene. Lucene is awesome for search in general, filters, and obviously full-text search. Very mature and well supported by so many big names and amazing engineers. So we take advantage of that but we had to change a few things to make it work perfectly for Vector use-case. We basically think Vector should be the main data type as it is the most difficult one to deal with. For instance, we modified Lucene to use X number of CPU / threads to build a single segment index. As a result, if/when needed, we can utilize hundreds of CPUs to index quicker and generate less number of segments that will enable lower query latency. We also built a custom File System Directory for Lucene to work off of GCS directly (or S3 later on). It can by-pass the kernel, read from network and write directly into the memory... no SSD, no page-cache, no mmap involved. Perhaps I should not say more...

We shard the data and index on about 6 x n2-standard-96 spot instances so the total cost of indexing the entire deep1b is less than $12. We are working on to make it $6. We separate indexing and query VMs. For queries we use dedicated VMs. USearch numbers look great and are better than ours if you run the query and indexing on the same VM/node. We believe design-wise distributed, task-oriented design is the right way to handle vector search for thousands of tenants with different size datasets. Data ingest is also a separate task for us so Ingest, Index and Query are all handled by different cluster of VMs.

Vectroid is pure Java solution based on modified version of Lucene. We use a custom built FileSystem to work directly with GCS (Google cloud object store). It is a terraform/helm managed Kubernetes deployment.

I think we have so many of those nice open source libraries but the problem is not the library or the algorithm (hsnw or ivf derivatives).. the problem is figuring out the right distributed architecture to balance cost, accuracy (recall) and speed (latency). I believe no single library will give you all that. For instance if you don't separate writes (indexing) from reads (queries) and scale them separately then your indexing will either suck or your indexing will destroy your read latency. You won't be able to scale as easily either. I believe that is why AWS created Aurora and Google Cloud created AlloyDB to scale relational databases (mysql/postgresql) by separating the reads/writes, implementing a scalable storage backend and by offloading a lot of shared works (replication, compaction, indexing) to cluster of machines.

No special hardware. Google Cloud vms. We use multiple of them during index building.

We just added Monero today!

Now Monero is supported!

This is Talip@Evercoin. We have mainly two kinds of fees. Miner/Network fees, which are fixed and can be listed (and we will). Second one is our fee, which is dynamically calculated based on multiple parameters such as our reserve, market trend and recent transactions. It is basically a risk calculation. We can and should say that on our FAQ but not sure if that is going to be 'transparent' enough. Btw, we are a new service, we need feedback from the community to know how to make our service better and time to get them done. We are committed to become better. So please keep the feedback coming.

This is Talip@Evercoin. We will be adding more coins over time starting with Monero in a week. We think support matters a lot. We want to make sure we are quickly reachable via anonymous chat on the website when needed.

This is Talip@Evercoin. Yes we are currently working on adding Monero. Should be ready in a week. Thanks for the feedback.

Locations: Palo Alto, CA | New York Position: Senior Solutions Architect (fulltime) Company: Hazelcast, Inc. (http://www.hazelcast.com) Hazelcast is an in-memory data grid solution.

As a Sn. Solutions Architect, you will work directly with customers and users to help them understand how they can use Hazelcast to achieve better performance and reliablity. Sn. Solutions Architect is an Hazelcast evangelist who loves giving trainings, talking at the conferences, being active in the community and consulting on distributed computing problems.

Responsibilities

Work with Sales personnel to provide technical support and develop proof of concept applications. Give trainings, speak at the trade shows and conferences. Be active in the community. Blog and tweet about Hazelcast features/tips/news. Collect feedback from the community, users and customers, shape the future of Hazelcast. Provide architectural and advisory consulting for key accounts to ensure successful implementations.

Requirements

10+ years in software development. Expertise in distributed caching, scalability and highly available systems. Strong technical skills in Java, Spring, Hibernate and JEE. Comfortable with Linux, Solaris, Unix and Windows. ~25% travel.

Please contact careers at hazelcast dot com.