Best Practices for Data Pipelines
To set up reliable data pipelines, there are some best practices you should follow:
1. Think about which data connectors you need
2. Abilities of your team (set up time, maintenance time, skills)
3. Budget
4. Customer support SLAs
One tool may not meet all requirements or needs, but if it’s best-in-class at what it does, it’s worth it. Some helpful advice I’ve seen online:
Safeguard your raw data. Keep your raw data in a separate database within your data warehouse. Don’t give write access to this data to anyone; it should be strictly read-only. The raw data database is storage for all the data from various sources and serves as a backup in case data modeling leads to any issues such as deletions.
-- Vinesh founder @ Bracket https://usebracket.com
(
no comments
| 2 points
| on HN
)
There are no comments yet.