Python Data Engineer

Your potential, unleashed. India’s impact on the global economy has increased at an exponential rate and Deloitte presents an opportunity to unleash and realize your potential amongst cutting edge leaders, and organizations shaping the future of the region, and indeed, the world beyond. At Deloitte, your whole self to work, every day. Combine that with our drive to propel with purpose and you have the perfect playground to collaborate, innovate, grow, and make an impact that matters. The Team Deloitte’s Technology & Transformation practice can help you uncover and unlock the value buried deep inside vast amounts of data. Our global network provides strategic guidance and implementation services to help companies manage data from disparate sources and convert it into accurate, actionable information that can support fact-driven decision-making and generate an insight-driven advantage. Our practice addresses the continuum of opportunities in business intelligence & visualization, data management, performance management and next-generation analytics and technologies, including big data, cloud, cognitive and machine learning. Work you’ll do Total Experience - 7 to 11 Years Relevant Experience - 5+ Location - Bangalore Roles and Responsibilities: The Data Engineer will work on data engineering projects for various business units, focusing on delivery of complex data management solutions by leveraging industry best practices. They work with the project team to build the most efficient data pipelines and data management solutions that make data easily available for consuming applications and analytical solutions. A Data engineer is expected to possess strong technical skills. Experience: - Knowledge of and skills in various programming languages primarily Python Must have knowledge on Back-end frameworks, Pandas, Numpy, Matplotlib/Seaborn Thorough understanding of containers and functions. Deployment experience with Kubernetes (K8s) or Functions is highly desirable. Experience using Cloud Native CI/CD tools (Azure Pipelines/Circle CI/Jenkins X). Experience deploying workloads to AWS or Azure with strong knowledge and understanding of the cloud provider's API / associated services and infrastructure Experience in test driven development & writing of unit and integration tests is a must Knowledge of other Cloud (Google Cloud Platform, Cloudera etc), and Integration services technologies is highly desirable. Experience working in agile teams with demonstrated application of the principles. Demonstrable proficiency in developing complex JavaScript applications. Experience of working with lean startup/agile development methodologies Strong analytical, problem-solving, and troubleshooting skills. Experienced with modern coding, testing, debugging and automation techniques. Rave about the benefits of CI/CD Have a high bar for user experience and quality. You are data driven and customer obsessed. Good communication skills. Key Characteristics Availability to solve production issues/taking ownership of interfaces deployed on prod etc Understanding in RDBMS and NoSQL databases (e.g., PostgreSQL, MongoDB). Ability to troubleshoot and resolve database issues. In-depth knowledge and experience in RESTful API interfaces. Comprehensive understanding of ETL processes from end-to-end. Technology champion who constantly pursues skill enhancement and has inherent curiosity to understand work from multiple dimensions. Interest and passion in Big Data technologies and appreciates the value that can be brought in with an effective data management solution. Has worked on real data challenges and handled high volume, velocity, and variety of data. Is proficient in designing data integrations and data processes across different types of architecture patterns using native cloud services, or custom application codes Excellent analytical & problem-solving skills, willingness to take ownership and resolve technical challenges. Contributes to community building initiatives like CoE, CoP. Mandatory Skills Programming Languages: Proficiency in Python and PySpark is paramount. Should have knowledge of data processing using Pandas, Numpy and relevant libraries to handle data. Databases: Strong knowledge of SQL and NoSQL databases Big Data Technologies: Experience with frameworks like Apache Spark, Hadoop, and Kafka for processing large datasets Cloud Computing: Familiarity with cloud platforms like Azure(preferred)/AWS(good to have) and their data services (e.g., Data Factory, Databricks, Synapse, Redshift) ETL/ELT & Data Warehousing: Expertise in designing and implementing ETL (Extract, Transform, Load) processes, and data modelling for efficient data warehousing Data Pipelines: Ability to develop, optimize, and maintain robust and scalable data pipelines Should have a good understanding of optimization techniques and able to resolve performance bottlenecks Knowledge of web frameworks and API is beneficial How you’ll grow At Deloitte, our professional development plan focuses on helping people at every level of their career to identify and use their strengths to do their best work every day. From entry-level employees to senior leaders, we believe there’s always room to learn. We offer opportunities to help build world-class skills in addition to hands-on experience in the global, fast-changing business world. From on-the-job learning experiences to formal development Programs at Deloitte University, our professionals have a variety of opportunities to continue to grow throughout their career. Explore Deloitte University, The Leadership Centre. Benefits At Deloitte, we know that great people make a great organization. We value our people and offer employees a broad range of benefits. Learn more about what working at Deloitte can mean for you. Our purpose Deloitte is led by a purpose: To make an impact that matters . Every day, Deloitte people are making a real impact in the places they live and work. We pride ourselves on doing not only what is good for clients, but also what is good for our people and the communities in which we live and work—always striving to be an organization that is held up as a role model of quality, integrity, and positive change. Learn more about Deloitte's impact on the world Interview tips We want job seekers exploring opportunities at Deloitte to feel prepared and confident. To help you with your interview, we suggest that you do your research: know some background about the organization and the business area you’re applying to. Check out recruiting tips from Deloitte professionals.