EngineeringStuf

$0.00051192 per M/tok ?

Assuming a 64 card cluster

swarmfile was built to deal with this type of problem

We built similar at Swarmfile, and it's compatible with Git too

I really do agree.

I was a lead Technical Architect and authority on behalf of HM Treasury for a while, and I will tell you this: this is just the tip of the iceberg in government procurement.

I've witnessed faulty systems in DVLA, DEFRA, DWP, Home Office, MOJ and Scottish Government. Systems that have directly resulted in suicide, false convictions, corruption and loss of money to the public purse.

The problem with Horizon and Fujitsu is that in the end the government has to sign it off, and there will be someone who is the Accountable Officer (AO). More often than not, all parties (customer and supplier) become incredibly motivated to protect the AO because it protects profits, protects reputational damage and essentially builds a good news story around the whole thing.

It's just elitism, wrapped up in cronyism, veiled in lies so that AOs can fail upwards into positions with suppliers. I've seen it too many times and I'm fed-up with it. Government is completely and utterly corrupt.

I used to work in Spend Controls in Cabinet Office, so I got to see this across about 20 UK Government departments.

It's a bit of a race to the bottom regarding outsourcing, there are good companies and there are good technologist's but they never last more than a few years due to poor contracts, poor decision-making and poor pay.

I've actually seen some very good companies deliver on time and under budget, but then the company fails because they did the job too well and follow-up work wasn't needed.

I think that for outsourcing companies "wage theft" and doing just enough to meet the contract are core components of their business. That is, they need to optimise for high fees, low pay and the bare-minimum in quality.

Yes, absolutely. Although that would require care and forethought regarding unintended consequences.

Most roads will lead back to consultancies stepping-in because the government org will likely lack the skills. And it's very easy for a consultancy to claim they're experts, but throw an army of graduates at it.

Interoperability between rules based systems (and legacy) software need to be considered too. It's a massive problem that will grow with time... Most government departments have some form of every generation of technology.

I've worked on a variety of large UK government systems for the past ten years.

This blog encapsulates the problem of writing government services/software, which often results in strange outcomes.

Writing software for government is essentially the codification of centuries worth of Acts of Parliament.

Now imagine building the HMPO passport system, and then some underlying Law/Act is changed or repealed etc.

Now someone has to find and change everything that the Law/Act affected in all systems.

Now consider that the government frequently outsources this work to expensive consultancies who are motivated to elongate contracts and extract maximum value from the client... And ideally become entrenched.

All whilst building systems of varying quality and inflexibility so that the next time that a Law/Act is changed then this whole process repeats.

There is no central decision making authority to wrangle this problem (there used to be Spend Controls), which is why Government services delivery is so expensive.

Yes, but you could optimise the generated questions over time to reduce cache-misses.

The number of chunks would be the same regardless of either approach.

The generation of questions can be done out-of-band by a cheaper model.

Their current implementation approach seems to require some computation per request. It would be a balance to see which strategy provides the most value.

The speed of responses overall would be faster.

Am I correct in reading that the RAG pipeline runs in realtime in response to a user query?

If so, then I would suggest that you run it ahead of time and generate possible questions from the LLM based on the context of the current semantically split chunk.

That way you only need to compare the embeddings at query time and it will already be pre-sorted and ranked.

The trick, of course, is chunking it correctly and generating the right questions. But in both cases I would look to the LLM to do that.

Happy to recommend some tips on semantically splitting documents using the LLM with really low token usage if you're interested.

I'm surprised that Nokia found out through the keynote presentation from Steve Jobs. LG and Prada announced their phone a little earlier and it had been shown already at the IF Design Awards a few months earlier.

Google "LG Prada Phone" for the Wikipedia article.

If Nokia had paid attention to those design awards then they too could have moved quickly on a similar device.

Is this a case where Nokia thought they had a moat?

I'm surprised that Nokia found out through the keynote presentation from Steve Jobs. LG and Prada announced their phone a little earlier and it had been shown already at the IF Design Awards a few months earlier.

https://en.m.wikipedia.org/wiki/LG_Prada

If Nokia had paid attention to those design awards then they too could have moved quickly on a similar device.

Is this a case where Nokia thought they had a moat?

I'm not sure why no one else has mentioned this, but "STOP" will only stop messages relating to that particular campaign. By sending "STOP ALL" it will stop messages from any campaign on that number.

I'm surprised to learn that this isn't an open standard. E.g I must make an API call to a browser vendor. Is this a form of vendor lock-in?