Data Engineer

Role Description: Data Engineer – Big Data The ideal candidate is a hands-on technology developer with experience in developing scalable applications and platforms. They must be at ease working in an agile environment with little supervision. The person should be a self-motivated person with a passion for problem solving and continuous learning
Role and responsibilities
• Strong technical, analytical, and problem-solving skills
• Strong organizational skills, with the ability to work autonomously as well as in a team-based environment
• Data pipeline framework development
Technical skills requirements The candidate must demonstrate proficiency in,
• CDH On-premise for data processing and extraction
• Ability to own and deliver on large, multi-faceted projects
• Fluency in complex SQL and experience with RDBMSs
• Project Experience in CDH experience, Spark, PySpark, Scala, Python, NiFi, Hive, NoSql DBs)
• Experience designing and building big data pipelines
• Experience working on large scale, distributed systems
• Experience working on any Databricks would be added advantage
• Strong hands-on experience of programming language like PySpark, Scala with Spark, Python.
• Exposure to various ETL and Business Intelligence tools
• Experience in shell scripting to automate pipeline execution.
• Solid grounding in Agile methodologies
• Experience with git and other source control systems
• Strong communication and presentation skills
Nice-to-have skills
• Certification in Hadoop/Big Data – Hortonworks/Cloudera
• Databricks Spark certification
• Unix or Shell scripting
• Strong delivery background across the delivery of high-value, business-facing technical projects in major organizations
• Experience of managing client delivery teams, ideally coming from a Data Engineering / Data Science environment
Qualifications
• B.Tech./M.Tech./MS or BCA/MCA degree from a reputed university

---