Senior Data Engineer

2 weeks ago

Kolkata, India UST Full time

Senior Data EngineersExperience - 6 +Yrs (5-7 relevant years) Location: TrivandrumMandatory Skill - Pyspark, SQL, Azure DatabricksUK Shift TimingJob Description Key Accountabilities / Responsibilities:Provide technical direction and leadership to data engineers on data platform initiatives - ensuring adherence to best practices in data modelling, End to End pipeline design, and code quality.Review and optimize PySpark, SQL, and Databricks code for performance, scalability, and maintainability.Offer engineering support and mentorship to data engineering teams within delivery squads, guiding them in building robust, reusable, and secure data solutions.Collaborate with architects to define data ingestion, transformation, and storage strategies leveraging Azure services such as Azure Data Factory, Azure Databricks, Azure Data LakesDrive automation and CI/CD practices in data pipelines using tools such as Git, Azure DevOps, and DBT (good to have).Ensure optimal data quality and lineage by implementing proper testing, validation, and monitoring mechanisms within data pipelines.Stay current with evolving data technologies, tools, and best practices, continuously improving standards, frameworks, and engineering methodologies.Troubleshoot complex data issues, analyse system performance, and provide solutions to development and service challenges.Coach, mentor, and support team members through knowledge sharing sessions, technical reviews. Required Skills & Experience:Developer / engineering background of large-scale distributed data processing systems (or experience in equal measure). Can provide constructive feedback based on knowledge.Proficient in designing scalable and efficient data models tailored for analytical and operational workloads, ensuring data integrity and optimal query performance.Practical experience implementing and managing Unity Catalog for centralized governance of data assets across Databricks workspaces, including access control, lineage tracking, and auditing.Demonstrated ability to optimize data pipelines and queries using techniques such as partitioning, caching, indexing, and adaptive execution strategies to improve performance and reduce costs.Programming in Pyspark (Must), SQL(Must), Python (Good to have).Experience with Databricks (Mandatory) and DBT (Good to have) is requiredImplemented cloud data technologies on either Azure (Must) other optional GCP, Azure or AWS.Knowledge around shortening development lead time and improving data development lifecycleWorked in an Agile delivery framework.

  • Senior Data Engineer

    1 week ago

    kolkata, India beBeeDataEngineer Full time

    Unlock Data PotentialAs a seasoned Senior Data Engineer, you will design and develop robust data pipelines in Python for ingesting and processing large-scale text datasets. Your primary mission is to transform massive datasets into high-quality formats.Data Engineering: Design, build, and own automated data pipelines in Python for processing terabyte-scale...

  • Senior Data Engineer

    3 weeks ago

    Kolkata, India Tata Consultancy Services Full time

    Greetings from TCS! Job Title: Senior Data Engineer Required Skillset: Python, Spark, Databricks, AWS (S3, Glue, AirFlow, Cloudwatch, Lambda) Experience Range: 8+ years Job Description ROLE AND MISSION DESCRIPTION Main tasks Master Databricks tools (job creation, cluster, notebook) and be able to query efficiently with SQL Maintain the platform in...

  • Senior Data Engineer

    3 weeks ago

    Kolkata, India Tata Consultancy Services Full time

    Greetings from TCS! Job Title: Senior Data Engineer Required Skillset: Python, Spark, Databricks, AWS (S3, Glue, AirFlow, Cloudwatch, Lambda) Experience Range: 8+ years Job Description ROLE AND MISSION DESCRIPTION Main tasks - Master Databricks tools (job creation, cluster, notebook) and be able to query efficiently with SQL - Maintain the platform in...

  • Senior Data Engineer

    7 days ago

    kolkata, India Tata Consultancy Services Full time

    Greetings from TCS Job Title: Senior Data Engineer Required Skillset: Python, Spark, Databricks, AWS (S3, Glue, AirFlow, Cloudwatch, Lambda) Experience Range: 8+ years Job Description ROLE AND MISSION DESCRIPTION Main tasks Master Databricks tools (job creation, cluster, notebook) and be able to query efficiently with SQL Maintain the platform in operational...

  • Senior Data Engineer

    3 weeks ago

    Kolkata, India Tata Consultancy Services Full time

    Greetings from TCS! Job Title: Senior Data Engineer Required Skillset: Python, Spark, Databricks, AWS (S3, Glue, AirFlow, Cloudwatch, Lambda) Experience Range: 8+ years Job Description ROLE AND MISSION DESCRIPTION Main tasks Master Databricks tools (job creation, cluster, notebook) and be able to query efficiently with SQL Maintain the platform in...

  • Senior Data Engineer

    4 days ago

    kolkata, India Mitra AI Full time

    We are seeking a skilled and proactive Senior Engineer with strong Snowflake expertise and AWS cloud exposure to lead the implementation and operational excellence of a scalable, multitenant modern data platform for a leading US-based marketing agency serving nonprofit clients. This role requires hands-on experience in managing Snowflake environments,...

  • Senior Data Engineer

    1 week ago

    Kolkata, India CXC Full time

    **Please apply only if you are available to work in Australian time zone and comfortable with 6 months contract duration**About the RoleWe’re seeking a highly skilled and autonomous Senior Data Engineer with strong experience in Power BI implementations to join our new data and analytics team. This role is critical to the development and enhancement of...

  • Senior Data Specialist

    1 week ago

    kolkata, India beBeeData Full time

    Senior Data Engineer PositionThe role of Senior Data Engineer involves designing, developing and maintaining web scraping scripts to automate data collection from various online sources.Key responsibilities include extracting, cleaning, structuring and storing large datasets in MySQL databases.Developing tools for data visualization and analysis of product...

  • Senior Data Engineer

    14 hours ago

    Kolkata, West Bengal, India Tata Consultancy Services Full time

    Greetings from TCSJob Title: Senior Data EngineerRequired Skillset: Python, Spark, Databricks, AWS (S3, Glue, AirFlow, Cloudwatch, Lambda)Experience Range: 8+ yearsJob DescriptionROLE AND MISSION DESCRIPTIONMain tasksMaster Databricks tools (job creation, cluster, notebook) and be able to query efficiently with SQLMaintain the platform in operational...

  • Data Engineering Expert

    1 week ago

    kolkata, India beBeeData Full time

    Senior Data Engineer - Data Acquisition RoleAbout the PositionWe are seeking an accomplished Senior Data Engineer to spearhead our data engineering team. The ideal candidate will possess extensive hands-on experience in building and optimizing data ingestion and transformation pipelines using Azure Data Platform, Databricks, Snowflake, and Kafka.Achieve...