1. Design and Maintenance of Data Pipelines: Develop, construct, test, and maintain architectures such as databases and large-scale processing systems.
2. ETL Development: Design and implement efficient, scalable ETL processes to improve data accessibility and usability.
3. Data Management: Ensure data quality and integrity by implementing proper data governance frameworks.
4. Collaboration: Partner with data scientists, analysts, and other stakeholders to understand data needs and deliver solutions.
5. Performance Tuning: Optimize and tune data systems for greater functionality in support of end-user needs.
6. Monitoring and Troubleshooting: Develop monitoring processes to track the performance and availability of data systems, responding to incidents and troubleshooting issues.
7. Documentation: Maintain clear and detailed documentation of all data processes and systems.
1. Extensive experience with SQL and NoSQL databases (e.g., MySQL, PostgreSQL, MongoDB).
2. Proficiency in programming languages such as Python, Java, or Scala.
3. Familiarity with big data technologies like Hadoop, Spark, and Kafka.
4. Experience with cloud computing services, particularly AWS, Azure, or Google Cloud.
5. Knowledge of data warehousing solutions such as Redshift, Snowflake, or BigQuery.
6. Experience with workflow management tools such as Apache Airflow or similar. 7. Knowledge in data architecture and data modelling
1. Problem-Solving: Strong analytical skills to solve complex problems and drive data-driven decisions.
2. Communication: Excellent communication skills to articulate complex data concepts to a variety of stakeholders.
3. Collaboration: Proven ability to work well in a team-oriented environment with cross-functional teams.
4. Attention to Detail: Keen attention to detail in data analysis and troubleshooting.
5. Time Management: Ability to manage multiple priorities and projects in a fast-paced environment
1. Bachelor’s degree in Computer Science, Information Technology, or a related field
2. Minimum of 3-5 years of experience in data engineering or related roles.
3. Experience with managing databases and building scalable data architectures.
4. Fresh graduates are encouraged to apply.