Data Engineer
2 months ago
Job Description :
- 5 to 8 years in the build release management, software configuration, design, development, and cloud implementation.
- Data Engineer with expertise in Python, Scala, PySpark, AWS S3, Glue, Redshift, Lambda, DBT, Serverless, Snowflake, SQL, Datalake, and Data Pipeline.
- Leveraged AWS Glue for data cataloging, ETL job orchestration, and schema evolution.
- Expert knowledge in AWS Data Lake implementation and support (S3, Glue,DMS Athena, Lambda, API Gateway, Redshift)
- Handling of data related activities such as data parsing, cleansing quality definition data pipelines, storage and ETL scripts
- Experiences in programming language Python/Pyspark/SQL
- Experience with data migration with hands-on experience
- Experiences in consuming rest API using various authentication options with in AWS Lambda architecture orchestrate triggers, debug and schedule batch job using a AWS Glue, Lambda and step functions
- Understanding of AWS security features such as IAM roles and policies
- Exposure to Devops tools
- AWS certification in AWS is highly preferred
- Should develop and maintained scalable data pipelines using Python, Scala and PySpark to process and transform large volumes of structured and unstructured data.
- Should develop and maintained scalable and distributed data processing pipelines using Scala and Apache Spark, handling large volumes of data efficiently.
- Should Implement complex data transformations, aggregations, and analytics using Scala and Spark
- Collaborated with data scientists to integrate machine learning algorithms into Spark workflows, enabling advanced data analysis and predictive modeling.
- Optimized Spark jobs by tuning configuration parameters, optimizing data partitioning, and leveraging in memory caching techniques, resulting in significant performance improvements.
-
Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN Orangeskill Technologies Pvt Ltd Full timeJob Description :We are looking for a talented and experienced Data Engineer to join our team. You will be responsible for designing, building, and maintaining data pipelines, workflows, and infrastructure to support efficient data processing and analysis. The ideal candidate will have experience in workflow orchestration tools, database management, data...
-
Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN PureSoftware Pvt Ltd. Full timeWe are seeking a talented and experienced Azure Data Engineer to join our team. As an Azure Data Engineer,Mandatory Skills :- Databricks(PySpark, Scala)- Data Factory/Synapse- SQL DB and DW- Working knowledge on Git Roles and Responsibilities :Requirements :- 4+ years of experience working as a Data Engineer, with a focus on Azure cloud platform.- Good...
-
Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN Vysystems Full timeData Engineer Need short joinersJob Location : Bangalore - Manyata Tech ParkMode : Hybrid.Experience : 4-8 yearsJob Description : We are looking for a skilled Data Engineer with expertise in Azure Data Factory, Azure Databricks, PySpark, Snowflake, and SQL. The ideal candidate will play a key role in designing, building, and maintaining scalable data...
-
Data Quality Engineer
2 months ago
Anywhere in India/Multiple Locations, IN Praxis Consultants Full timeJob Summary :We are seeking a detail-oriented and analytical Data Quality Engineer to join our team. The ideal candidate will be responsible for ensuring the accuracy, integrity, and reliability of our data across various systems. You will work closely with data engineers, data scientists, and other stakeholders to implement data quality processes and...
-
DevOps & Data Engineer
6 months ago
Anywhere in India/Multiple Locations, IN Dotflick Solutions Full timeProfile : DevOps and Data EngineerJob description : This role is a Hybrid DevOps and Data Engineering.Mode : WFO/WFHRoles & responsibilities : - Setting up tools and required infrastructure- Defining and setting development, test, release, update, and support processes for DevOps operation- Have the technical skill to review, verify, and validate the...
-
Data Engineer
1 month ago
Anywhere in India/Multiple Locations, IN Machine Minds Full timeJob Description : Data EngineerThe Position : We are seeking an experienced Data Engineer to join our team of passionate professionals working on cutting-edge technology. In this role, you will be responsible for the ELT of Canibuild data using Python, Airflow, and SQL within an AWS environment. Additionally, you will create and maintain data visualizations...
-
Data Engineer
1 month ago
Anywhere in India/Multiple Locations, IN Crescendo Global Full timeJob Description : Are you an experienced Data Engineer with expertise in Python, Pyspark, and AWS? Our client, a leading technology firm in India, is looking for a Senior Data Engineer to join their team. If you have a strong analytical mindset and a passion for leveraging data to drive business insights, this opportunity is perfect for you.Location : PAN...
-
Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN Risk Resources LLP Full timeOverview :The Data Engineer specializing in Snowflake, Azure, and AWS technologies plays a crucial role in designing, developing, and maintaining scalable data solutions to meet the organization's evolving data needs. The role requires expertise in a range of data engineering tools and platforms for managing and transforming large sets of structured and...
-
Lead Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN ARR Recruitment Solutions Full timeJob Description :Required skills & experience :- Proficiency in data modeling and database management.- Strong programming skills in Pyspark, Python and SQL.- Experience with ETL (Extract, Transform, Load) processes.- Familiarity with strong data warehousing and Azure cloud platform with immense working knowledge in Databricks. - Knowledge of data governance...
-
Technical Lead
2 months ago
Anywhere in India/Multiple Locations, IN Praxis Consultants Full timeJob Summary :We are looking for a highly skilled and experienced Technical Lead (Sr. Data Engineer/Data Architect / Data Modeler) to join our team. will play a crucial role in designing data architecture and models based on business requirements, and ensuring data integrity, security, and performance. Oversee technical aspects of data integration and...
-
Data Engineer
4 weeks ago
Anywhere in India/Multiple Locations, IN Apidel Technologies Full timeData Engineer | 100% Remote | 12-Month Contract. We are looking for an experienced Data Engineer to join our team on a 12-month remote contract. This role offers a great opportunity to work with Azure services, including Azure Databricks, Azure DataFactory, and Azure DevOps, to build secure, scalable data pipelines. What You'll Do :- Develop data...
-
Lead Data Engineer
4 weeks ago
Anywhere in India/Multiple Locations, IN Talentoj Full timeRequirements : - 6+ years' experience with Big Data Technologies such as Apache Spark, Cloud Native, and Data Lakes, and Databricks. - Delivered ETL/ELT solutions including data extraction, transformation, cleansing, data integration, and data management. - Implemented batch ingestion pipelines. - Strong Python skills - automate tasks, write unit tests...
-
Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN Huntsmen & Barons Full timeRole : Data Engineer - Spark/Scala. Job Description :- Relevant Experience 5 years.- The indent is created for Apache spark. But the actual demand is for Spark/ Scala.- Expertise in Data Engineering. - Hands-on experience in design and development of big data platform.- Deep understanding of modern data processing technology stacks: Spark, HBase, Hive and...
-
Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN Emorphis Technologies Pvt. Ltd. Full timeJob Description :- Senior Data Analyst/ engineer with Snowflake and Azure experience - 7+ Years of overall technology experience that includes at least 3+ years of software development, data engineering and architecture - Excellent understanding of business functions, KPIs and Metrics and Translate them into data analytic solutions, architecture, costing...
-
Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN Huntsmen & Barons Full timeRole : Data Engineer - Apache spark,Spark, Scala, PYspark, Hadoop, Hbase, HiveLocation : Pan IndiaWork Mode- HybridYear of Experience :Minimum 5 Years tp 12 YearsJob Description :- Relevant Experience 5 years.- The indent is created for Apache spark. But the actual demand is for Spark/ Scala.- Expertise in Data Engineering.- Hands-on experience in design and...
-
Data Engineer
4 weeks ago
Anywhere in India/Multiple Locations, IN DoublU Pty Ltd Full timeJob Overview :We are seeking a skilled GCP Data Engineer to join our dynamic data team. The ideal candidate will be responsible for designing, building, and maintaining scalable data pipelines and architectures in Google Cloud Platform (GCP). You will work closely with data scientists, analysts, and other stakeholders to ensure that data is accessible,...
-
Big Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN D-TechWorks Pvt Ltd Full timeWe are looking for Aws big data engineer In Remote , India Position : Aws big data engineerExperience : 7 to 9 YearLocation : Remote , India Mandate skills : BigData Python Pyspark AWS Glue Airflow Redshift Responsibilities :- Ensure effective Design, Development, Validation and Support activities in line with client needs and architectural requirements.,-...
-
Python Developer
2 months ago
Anywhere in India/Multiple Locations, IN Red Oak Technologies Full timeRed Oak Technologies is a leading provider of comprehensive resourcing solutions across a variety of industries and sectors including IT, Marketing, Finance, Business Operations, Manufacturing and Engineering. We specialize in quickly acquiring and efficiently matching top-tier professional talent with clients in immediate need of highly skilled contract,...
-
Azure Data Engineer
1 month ago
Anywhere in India/Multiple Locations, IN Scaling Theory Technologies Pvt Ltd Full timeRoles and Responsibilities :- Information Model object build/enhancements (PO, Invoice, SES, RO, Vendor, etc) based on functional design- Technical/Functional validation of IM objects- Troubleshooting issues with code and implementing fixes- Support daily data loads from SAP/NonSAP systems into Information ModelTechnical Skills :- SQL Expert - Build [must...
-
Java Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN Hum Technologies Full timeJob Description : We are hiring for Sr. Java Data Engineer with strong Exp. in Java, Spring, Restful-API, Microservices, AWS / AZURE, SQL, Data Structures, Algorithms, Multithreading, Hashmaps, Arrays.Company Name : Hum TechnologiesClient : One of the Fortune 500 CompaniesLocation : Remote/ HybridRole : Java Data Engineer.Must-Have :- 4+ years of experience...