Complete your Databricks User Groups profile!

Fill out a few details about yourself so the community can get to know you.
Hyderabad Databricks User Group

Transitioning to a Data Engineering Role: Seeking Advice from the Community

Summary: Syed Mohammed Sylani is new to the field and is focusing on developing problem-solving skills using platforms like LeetCode. They are seeking insights on the value of coding challenges for becoming a data engineer and are open to additional resources and guidance. Sridhar Pothamsetti recommends Databricks courses and provides links to resources for practice and projects, including analytics projects that can be completed in a short time. Syed Mohammed Sylani expresses appreciation and requests suggestions for minor projects to gain practical experience.
AI Summary

Hi everyone!

As someone who's new to the field and currently focusing on developing problem-solving skills through platforms like LeetCode, I'm curious to hear from those of you with experience. How valuable have you found coding challenges in your journey to becoming a data engineer? Are there other resources or activities you'd recommend to complement this experience? What key skills or knowledge areas should I focus on to better prepare for potential trainee roles?

Additionally, if anyone knows of any trainee positions or opportunities within the data engineering field, please feel free to share. Any guidance or shared experiences would be incredibly helpful!

Looking forward to your insights!

5 comments
Sridhar Pothamsetti
3 weeks, 5 days ago

The exercises you do will really help. you can find courses of databricks at https://customer-academy.databricks.com/

Also you can practice at https://docs.databricks.com/aws/en/getting-started/free-edition

Syed Mohammed Sylani
3 weeks, 5 days ago

Thank you so much for your support and guidance!

Syed Mohammed Sylani
3 weeks, 5 days ago

Also, Sridhar, if you have any minor projects that I could work on to understand how things actually work in a company, I’d be grateful if you could share them. Any project suggestions or guidance would also be really helpful. Thank you!

Sridhar Pothamsetti
3 weeks, 3 days ago

Databricks Demo Center Library | Databricks

Tutorial: 3 Free Databricks Analytics Projects You Can Do In An Afternoon | Databricks Blog

Riyaz Ali Mohammad
3 weeks, 3 days ago

Hello Syed,

IMO, I would suggest you to focus on SQL & Python in a broader manner:
- SQL : Writing optimized SQL goes a long way, knowing how the query would be parsed by the engine would make you a better Data Engg.

- Databases: Get to know how stuff works in a postgres (for e.g.) based databases and cover the ground (stored procedures, functions, DML, DDLs so on..)

- Python: Start with general scripting and move towards regexp/pattern matching, frameworks like Pandas, PySpark

- Understanding of distributed systems: Hadoop/Spark

- Understanding of AI Basics: LLMs, skills, MCP, memory management

Syed Mohammed Sylani
3 weeks, 6 days ago