andrewmcewen
andrewmcewen on Hacker News
That's a great point, we think of Secoda as _both_ a search and curation tool. The search portion of the tool is accomplished through the no-code integrations. When you connect Snowflake, for example, we extract metadata about the tables in Snowflake such as the columns, descriptions, number of queries run, people who are frequently using that table, etc. All that information becomes searchable in the catalog. After a company integrates all of their data sources, we see that teams leverage the curation capabilities of the product. Editors can add documentation through our documentation editor to provide additional context about the data resources that are discoverable in Secoda. In addition, teams can add shared definitions of metrics, answer questions that are asked by data consumers, and create ad hoc analyses that are also discoverable.
Thank you! Secoda is different from DataHub in a few ways:
1. If you're using the DataHub open source solution it requires a data engineer to get the platform up and running and maintained, which can be a fairly expensive cost depending on the salary of the data engineer. Secoda has 15+ no code integrations that can be setup in 5 minutes and is a fully managed solution. We are releasing a metadata API that will be available before the end of the year, in case an organization is using a product that we do not currently integrate with.
2. Acryl (managed version of DataHub) is mainly focused on the data catalog, which they do a great job for. However, they don't provide the questions, dictionary, and visualization components that we provide in addition to the catalog. These additional components of the product add more context around data knowledge, and are also focused on helping non-technical users understand company data. Whereas the data catalog is focused more on helping technical data users understand company data.
3. Also if you're using Acryl, you'll have to get in touch with their team to get a demo of the product. For Secoda, you can signup at https://app.secoda.co and try out a free trial of the product without having to talk with our team. We do offer demos if people are interested though.
Secoda is different from ThoughtSpot in a few ways:
1. It is likely that you'll need to setup a 1:1 meeting with a ThoughtSpot expert to help get your company up and running on the software. Our goal with Secoda is to be a self-service platform that is designed for any company, large or small, to get their data knowledge base setup in 5 minutes.
2. ThoughtSpot's price point is typically in the six figure range, which is much higher than Secoda's price point that starts at $29/editor on the platform.
3. Thoughtspot's core focus is providing answers to data questions through visualizations. Secoda takes a more comprehensive approach to documenting data knowledge. In addition to having visualizations that help answer questions, we also provide a shared data dictionary for defining metrics, as well as a catalog that can store tables, dashboards, jobs, and many other data resources.
An example of what lineage would look like is the following: You have source tables A and B that are joined to create a model C and then C is used in dashboard D. We are able to infer the lineage A->C->D and B->C->D.
We extract lineage in a couple of different ways. The main way is by parsing SQL queries in your data warehouse to determine which tables and dashboards are upstream/downstream. The other way we extract lineage information provided directly from dbt and BigQuery who have nice APIs for this information.
We are working releasing an API in Q4 that supports pushing information from say an Airflow DAG to Secoda to give us more lineage context. Hopefully this answers your questions.
Thanks for the positive message. We appreciate the support from people who understand the problem that we're trying to solve with Secoda. We haven't had someone request Google Data Studio as an integration, but if you were interested in discussing what that would look like please reach out to
[email protected]. It looks like there's an API for accessing assets in Google Data Studio, so it's definitely possible.
Our most requested feature is discussion threads attached to each data resource (table, dashboard, etc) to build context around a resource. So that will be coming in the near future, and we are happy to hear you also think it would be super helpful!
You've hit the nail on the head here. Long term we want to bridge the gap between the data team and rest of the organization through a central repository of data knowledge. Using MDM's terms the data team = IT, the rest of the organization = business, and repository of data knowledge = master data. So it's almost a one-to-one comparison, and we've never heard of MDM so this is great! We are also trying to make the experience lightweight and user friendly so that everyone is interested in exploring their company's data.
The catalog portion of the Secoda product is similar to Amundsen, but we also have a Data Dictionary for defining metrics, Analysis Documents for queries and charts, and Requests for handling data questions. We take these different pieces of functionality and make everything interconnected, so that it's one unified repository for your data knowledge.
Additionally, we try and make Secoda easy to use for both technical and non-technical users, whereas a tool like Amundsen is more focused on the technical user.
The main way that Secoda differs from a traditional reporting tool is that we offer a more complete view into the data knowledge/context of an organization. A traditional reporting tool like Looker does great for reports, but it misses context about how certain models are created, where data is coming from downstream, and more general knowledge that is stored inside of a wiki. Secoda takes all of the context across a data stack and puts it in one central place. So we actually see ourselves as complementary to these reporting tools as we are a layer on top of them.
You can add tags to any data resource in Secoda (table, column, dashboard, dictionary term, etc). When you tag resources, you can search for them by tag. Our customers have found it very useful for keeping everything organized.
Thanks, we hope so too! At Secoda, we really want to make data less intimidating for anyone in the organization to explore and use. When the whole company can use data to help inform their decisions it really does make a big difference.
We have pre-built integrations with many popular data tools (Snowflake, dbt, Tableau, etc.) that can be setup in less than 5 minutes. If those don't fit your needs then we can definitely setup some time to discuss the API.
Hi there, I'm the CTO of Secoda so happy to answer your question. Our dbt integration works with dbt cloud, and are working on making it compatible with dbt core as well. What we'll pull from dbt via their API is the metadata associated with the models, docs, and, jobs. We have a free version of the platform, so you can sign up and test it out if you'd like to see what that information looks like in Secoda.