But to the point of comparison between these and tools like Traceloop - it's interesting to see this space and how each platform takes it's own path and finds its own use cases.
LangSmith works well within the LangChain ecosystem together with LangGraph, LangServe. But if you're using LlamaIndex, or even just vanilla OpenAI you'll be spending hours to set up your observability systems.
Braintrust and Humanloop (and to some extend other tools I saw in this area) take the path of "full development platform for LLMs".
We try to look at it as developers look at tools like Sentry. Continue working in your own IDE with your own tools (wanna manage your prompts in a DB or in git? Wanna use LLMs your own way with no frameworks? no problem). We install in your app, with one line and we work around your existing code base and make monitoring, evaluation and tracing work.
1. Traceloop Otel 2. Langtrace.ai Otel 3. OpenLIT Otel 4. Portkey 5. Langfuse 6. Arize LLM 7. Phoniex SDK 8. Truera LLM 9. Truelens 10. Context 11. Braintrust 12. Parea 13. Context AI 14. openlayer.com 15. Deepchecks 16. langsmith 17. Confident AI 18. Helicone 19. Langwatch.ai 20. Arthur 21. Aporia 22. scale.com 23. Whylabs 24. gentrace.ai 25. humanloop.com 26. fixpoint.co 27. W n B Traces 28. Langtail 29. Fiddler 30. Evidently Ai 31. Superwise 32. Exxa 33. Honeyhive 34. Flowstack 35. Log10 36. Giskard 37. Raga AI 38. AgentOps 39. Patronus AI 40. Mona 41. Bricks Ai 42. Sentify 43. LogSpend 44. Nebuly 45. Autoblocks 46. Radar / Langcheck 47. Dokulabs 48. Missing studio 49. Lunary.ai 50. Censius.ai 51. ML flow 52. Galileo 53. trubrics 54. Prompt Layer 55. Athina 56. getnomos.com 57. c3.ai 58. baselime.io 59. Honeycomb llm