Senior Data Engineer

3 weeks ago


Thiruvananthapuram, Kerala, India Quantiphi Analytics Solutions Private Limited Full time
Job Description

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.

If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi

Role: Senior Data Engineer (Databricks)

Experience Level: 3 to 6 Years

Work location: Mumbai, Bengaluru, Trivandrum

Role & Responsibilities:

- Design and implement data solutions (ETL pipelines, Processing and Transformation logic) using Snowflake as a key platform.

- Design Virtual warehouses which are optimized for performance and cost (auto scaling, idle time etc)

- Write SQL , stored procedures to implement business logic, other critical requirements like encryption, data masking etc

- Implement solutions with features like Snowpipe, tasks, streams,dynamic tables etc

- Design the solutions by leveraging key concepts around the architecture of Snowflake, its data distribution and partitioning mechanism, data caching etc.

- Identify queries which take more time , look at the logs etc and provide solutions to optimize those queries

Must have skills:

- 3+ years experience of Hands-on in data structures, distributed systems, Spark, SQL, PySpark, NoSQL Databases
- Strong software development skills in at least one of: Python, Pyspark or Scala.

- Develop and maintain scalable & modular data pipelines using databricks & Apache Spark

- Experience building and supporting large-scale systems in a production environment

- Experience working on either of the cloud storages - AWS S3, Azure data lake, GCS buckets

- Exposure to the ELT tools offered by the cloud platforms like, ADF, AWS Glue, Google dataflow

- Integrate Databricks with other cloud services like AWS, Azure or Google Cloud

- Implement and optimize Spark jobs, data transformations, and data processing workflows in Databricks

- Managing and configuring databricks environments, including clusters and notebooks

- Experience working with different file formats - parquet, databricks delta lake

- Experience working with data governance on databricks, implementing unity catalog, delta share

  • Thiruvananthapuram, Kerala, India Quantiphi Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.If working in an environment that encourages you to innovate and excel, not just in professional but personal life,...

  • Senior Data Engineer

    3 weeks ago


    Thiruvananthapuram, Kerala, India Quantiphi Full time

    Roles and Responsibilities ➔ Review and analyze existing data collection systems, and make recommendations on improvement, maintenance, or other factors to improve the system ➔ Supervising junior data engineering teams to ensure they carry out their duties in line with the business's goals ➔ Responsible for designing, deploying, and maintaining the...


  • Thiruvananthapuram, Kerala, India Quantiphi Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Roles and ResponsibilitiesReview and analyze existing data collection systems, and make recommendations on improvement, maintenance, or other factors to improve the systemSupervising junior data engineering teams to ensure they carry out their duties in line with the business's goalsResponsible for designing, deploying, and maintaining the organization's...


  • Thiruvananthapuram, Kerala, India Talent500 Full time

    Talent500 is hiring for one of its clients.About Pella India:ANSR has set up Pella India, a Global Capability Centre in Hyderabad. The center's mission is to create a high-impact, high-trust technology and data hub that strengthens Pella's global capabilities and accelerates innovation. Pella India operates as a direct extension of the Pella U.S. IT team to...


  • Thiruvananthapuram, Kerala, India Wider World Immigration Full time ₹ 36,00,000 - ₹ 48,00,000 per year

    Min Experience:4 YearsJob Location:Australia, Germany, Singapore, and CanadaSalary Range:70 to 80 LPA with other BenefitsRole DescriptionThis is a full-time on-site role for a Senior Data Scientist inAustralia, Germany, Singapore, and Canada. The Senior Data Scientist will be responsible for performing data analysis, creating data visualizations, and...

  • Data Engineer

    3 days ago


    Thiruvananthapuram, Kerala, India Trenser Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    OverviewThis position is for a Data Engineer for a team that works on a long-term software projectfor a US client. The job location is Trivandrum, India. He/ she will be communicating withthe client representatives in the USA, both for technical and management aspects.Communication will be through various modes of collaboration including webconferences,...

  • Data Engineer

    3 days ago


    Thiruvananthapuram, Kerala, India Quantiphi Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth. If working in an environment that encourages you to innovate and excel, not just in professional but personal life,...

  • Data Engineer

    5 days ago


    Thiruvananthapuram, Kerala, India Sentient Scripts Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    We are seeking a skilled and proactive Data Engineer with 3-6 years of hands-on experience in building robust data solutions on the Azure platform. The ideal candidate will have expertise in Azure Data Factory, Azure Data Lake, Azure Databricks, SQL, and Python, with additional experience in Power BI or Tableau considered a plus. You will be instrumental in...

  • Senior Data Scientist

    4 weeks ago


    Thiruvananthapuram, Kerala, India Blackstraw Full time

    We are looking for Senior Data Scientists to lead a data science team and help us gain useful insight out of raw data. Senior Data Scientist responsibilities include managing the data science team, planning projects and building analytics models. You should have a strong problem-solving ability and a knack for statistical analysis. If you're also able to...

  • Data Engineer

    3 weeks ago


    Thiruvananthapuram, Kerala, India Skywalk Global Full time

    Skywalk Global is on the lookout for Azure & Data ETL Developer to strengthen our Data Engineering team. Location: Trivandrum Experience: 3–6 Years Availability: Immediate to 30 days preferred What You'll Do: Develop & maintain Azure Functions (.NET) for data processing Build ELT pipelines with ADF / Azure Data Services Work on Snowflake, Azure...