Senior Dev Engineer- Data Platform

2 weeks ago


Pune, Maharashtra, India The Product Guys Full time
Job Description

Responsibilities include but are not limited to the following: 

● Optimize Spark clusters for cost, efficiency and performance by implementing robust monitoring systems to identify bottlenecks using data and metrics. Provide actionable recommendations for continuous improvement 

● Optimize the infrastructure required for extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS 'big data' technologies 

● Work with data and analytics experts to strive for greater cost efficiencies in our data systems 



Requirements

Who you are: 

● Experience with processing large workloads and complex code on Spark clusters 

● Proven experience in setting up monitoring for Spark clusters and driving optimization based on insights and findings 

● Experience in designing and implementing scalable Data Warehouse solutions to support analytical and reporting needs 

● Strong analytic skills related to working with unstructured datasets 

● Build processes supporting data transformation, data structures, metadata, dependency, and workload management 

● Working knowledge of message queuing, stream processing, and highly scalable 'big data' data stores 

Experience using the following software/tools: 

● Expertise with Python and Jupyter notebooks is a MUST. 

● Experience with big data tools: Spark, Kafka, etc. 

● Experience with relational SQL and NoSQL databases, including Postgres and Cassandra 

● Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc. 

● Experience with AWS cloud services: EC2, EMR, RDS, Redshift 

● Working knowledge of stream-processing systems: Storm, Spark-Streaming, etc. is a plus 



Requirements
We are seeking a Senior Network Operations Engineer who: ● Holds a degree in Computer Science or equivalent, with 10+ years of professional experience, including experience in an enterprise SaaS organization. ● Possesses expert-level knowledge in Firewall, IDS/IPS, DNS, BGP, OSPF, AWS, and Azure network constructs. ● Has experience deploying physical network and security devices, as well as cloud networking solutions like web application firewalls. ● Demonstrates proficiency in coding with Python, Java, Ruby, or equivalent languages. ● Familiarity with deploying cloud network infrastructure using Terraform and CloudFormation, or experience with Chef & Ansible. ● Exhibits confidence in project ownership and issue resolution. ● Has a passion for excellence, natural problem-solving skills, and critical thinking abilities. ● Possesses a solid understanding of networking systems and protocols, particularly in a public cloud environment. ● Thrives in a collaborative and supportive work environment. Desirable qualifications (not essential) include: ● Experience with monitoring technologies like Prometheus, Influx, Grafana, Elasticsearch, New Relic, Dynatrace or Datadog. ● Familiarity with container technologies such as Kubernetes, ECS, and EKS. ● Knowledge of databases like MySQL, MS-SQL, and NoSQL. ● Understanding of security and certificate management. ● Relevant operating system, networking, and cloud certifications.

  • Pune, Maharashtra, India Antal International Full time

    Senior Manager, Engineering - Software Architecture and Cloud Support Location: Pune Role Overview: My Client is a leading global product organisation based in the US, and have a state-of-the-art engineering center in Pune. They are looking for a Senior Manager of Architecture and Cloud Support, who will build, lead and manage a high-performance team focused...


  • Pune, Maharashtra, India TIAA Full time

    We are a dynamic organization that values innovation and excellence. As a Senior Platform Engineer - Data Science Expert, you will play a critical role in building end-to-end systems as a Platform Engineer, ML DevOps Engineer. You will possess functional knowledge of the interplay between Model Governance, Change and Release Management.You will be...


  • Pune, Maharashtra, India The Product Guys Full time

    Responsibilities include but are not limited to the following: ● Optimize Spark clusters for cost, efficiency and performance by implementing robust monitoring systems to identify bottlenecks using data and metrics. Provide actionable recommendations for continuous improvement ● Optimize the infrastructure required for extraction, transformation, and...


  • Pune, Maharashtra, India Deutsche Bank Full time

    **Job Summary**We are seeking a highly skilled and experienced Senior Engineer to join our team at Deutsche Bank. As a key member of our technology team, you will be responsible for designing and implementing innovative solutions that drive business growth and excellence.**Key Responsibilities**Collaborate with cross-functional teams to deliver high-quality...


  • Pune, Maharashtra, India HCLSoftware Full time

    Exp:6- 8 yrs only Location: Pune Send resume to: Please provide below details: Name Exp: CTC: ECTC: Availability for Face-to-Face on april 12th: Yes/no Current location: JD: The Data Platform & Modeling Engineer is responsible for both designing robust data models (including Data Vault) and building the underlying data infrastructure. This role bridges...


  • Pune, Maharashtra, India HCLSoftware Full time

    Exp:6- 8 yrs onlyLocation: PuneSend resume to: mounika-d@hcl.softwarePlease provide below details:NameExp:CTC:ECTC:Availability for Face-to-Face on april 12th: Yes/noCurrent location:JD:The Data Platform & Modeling Engineer is responsible for both designing robust data models (including Data Vault) and building the underlying data infrastructure. This role...

  • Senior Data Engineer

    2 weeks ago


    Pune, Maharashtra, India Atlas Copco Full time

    Passionate people create exceptional thingsDid you know that the solutions we develop are a key part of most industries Electronics medical research renewable energy food production infrastructure and many more We xc2 xb4re everywhere Working with us means working with the latest technologies and groundbreaking sustainable innovations With our...


  • Pune, Maharashtra, India Velotio Full time

    About Velotio : Velotio Technologies is a product engineering company working with innovative startups and enterprises. We have provided full-stack product development for 110 startups across the globe, building products in the cloud-native, data engineering, B2B SaaS, IoT & Machine Learning space. Our team of 400 elite software engineers solves hard...


  • Pune, Maharashtra, India SR analytics Full time

    Senior Data Engineer Job Title: Senior Data Engineer Location: Mundhwa, Pune Company: SR Analytics Work from Office About Us We at SR Analytics , a Data Analytics and Business Intelligence Services firm, are looking to add a Senior Data Engineer to our team. If you are passionate about Azure Cloud technologies and data engineering, have a strong...


  • Pune, Maharashtra, India Velotio Technologies Full time

    About Velotio TechnologiesWe are a product engineering company that works with innovative startups and enterprises. Our team of elite software engineers solves hard technical problems while transforming customer ideas into successful products.Job Description:We are seeking a highly skilled Data Platform Engineer to optimize our Spark clusters for cost,...

  • Senior Lead Engineer

    3 weeks ago


    Pune, Maharashtra, India NTT DATA Full time

    b Req ID 281257 NTT DATA Services strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a Senior Lead Engineer to join our team in Pune Mah xc4 x81r xc4 x81shtra IN-MH India IN Basic...

  • Senior Data Engineer

    4 weeks ago


    Pune, Maharashtra, India SPIRO Full time

    Position: Senior Data Scientist / Data Engineer Location: Pune Experience: 5+ years Job Summary: We are seeking an experienced Senior Data Scientist / Data Engineer with a strong background in PySpark, Spark, and Big Data technologies. The ideal candidate will have over five years of hands-on experience in the field, demonstrating a deep understanding of...


  • Pune, Maharashtra, India Barclays Full time

    Job DescriptionJoin us as a Infra Migration Engineer/Dev-Ops at Barclays, responsible for supporting the successful delivery of Location Strategy projects to plan, budget, agreed quality and governance standards. You&aposll spearhead the evolution of our digital landscape, driving innovation and excellence. You will harness cutting-edge technology to...


  • Pune, Maharashtra, India SRS Consulting Inc Full time

    Hello,We are hiring for " Salesforce Dev+Support Engineer "Exp:5+YearsLocs:RemoteShift Timings : 5PM to 2AMNotice period : Immediate joiners(Notice period served candidates)Mandatory Skills : Salesforce Development, Salesforce Support, Salesforce CPQ .NOTE: We are looking for Immediate joiners(Notice period served candidates).Apply only if you have 5+Years...

  • Senior Data Engineer

    4 weeks ago


    Pune, Maharashtra, India SPIRO Full time

    Position: Senior Data Scientist / Data EngineerLocation: PuneExperience: 5+ yearsJob Summary:We are seeking an experienced Senior Data Scientist / Data Engineer with a strong background in PySpark, Spark, and Big Data technologies. The ideal candidate will have over five years of hands-on experience in the field, demonstrating a deep understanding of data...

  • Senior Data Engineer

    2 weeks ago


    Pune, Maharashtra, India NCSI Technologies Private Limited Full time

    Job Description : Senior Data Engineer Experience : 6 to 12 Years Location : Hinjewadi Phase 1, Pune Requirements : - Designing and implementing data ingestion pipelines from multiple sources using Azure Databricks - Ensure data pipelines run smoothly and efficiently - Developing scalable and re-usable frameworks for ingesting data sets - Integrating the...


  • Pune, Maharashtra, India Epergne Solutions Full time

    Epergne Solutions is looking for Data Platform Engineer (Python + Airflow Data Engineer) Total Yrs. of Experience* 6+ yrs Relevant Yrs. of experience* 5+ yrs Mandatory skills* Python / Airflow Data Engineer Budget: As per market Standards Work Location* India Mode- Work from Office Positions: Contract Payroll: Epergne solutions Detailed JD *(Roles and...


  • Pune, Maharashtra, India Synechron Full time

    Synechron's progressive technologies and optimization strategies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud & Dev Ops, Data, and Software Engineering.We are committed to fostering an inclusive culture that promotes equality, diversity, and respect for all.We encourage applicants from diverse backgrounds to apply and offer flexible...


  • Pune, Maharashtra, India LTIMindtree Limited Full time

    Job DescriptionJob descriptionData Platform EngineerPrimary Skills- Apache NiFi Administration experience like building clusters- NiFi cluster version upgrades- Strong troubleshooting skills on linux- Good knowledge of kafka administration- Configuration management tools like Jenkins AnsibleSecondary Skills- Python automation skills- Database management like...


  • Pune, Maharashtra, India SRS Consulting Inc Full time

    Hello,We are hiring for "Salesforce Dev+Support Engineer"Exp: 5+YearsLocs: RemoteShift Timings: 5PM to 2AMNotice period: Immediate joiners(Notice period served candidates)Mandatory Skills: Salesforce Development, Salesforce Support, Salesforce CPQ .NOTE: We are looking for Immediate joiners(Notice period served candidates).Apply only if you have 5+Years of...