Lead Data Engineer
1 month ago
Roles and Responsibilities:
- Working with clients to understand their data.
- Based on the understanding you will be building the data structures and pipelines.
- You will be working on the application from end to end collaborating with UI and other development teams.
- You will be working with various cloud providers such as Azure & AWS.
- You will be engineering data using the Hadoop/Spark ecosystem.
- You will be responsible for designing, building, optimizing and supporting new and existing data pipelines.
- Orchestrating jobs using various tools such Oozie, Airflow, etc.
- Developing programs for cleaning and processing data.
- You will be responsible for building the data pipelines to migrate and load the data into the HDFS either on-prem or in the cloud.
- Developing Data ingestion/process/integration pipelines effectively.
- Creating Hive data structures, metadata and loading the data into data lakes / Bigdata warehouse environments.
- Optimized (Performance tuning) many data pipelines effectively to minimize cost.
- Code versioning control and git repository is up to date.
- You should be able to explain the data pipeline to internal and external stakeholders.
- You will be responsible for building and maintaining CI/CD of the data pipelines.
Preferred Qualifications:
- Bachelor’s degree in computer science or related field.
- Minimum of 5+ years working experience with Spark, Hadoop eco systems.
- Minimum of 4+ years working experience on designing data streaming pipelines
- Minimum experience of 3+ years on NoSQL and Spark Streaming.
- Proven experience with big data ecosystem tools such as Sqoop, Spark, SQL, API, Hive, Oozie, Airflow, etc..
- Solid experience in all phases of SDLC with 10+ years of experience (plan, design, develop, test, release, maintain and support)
- Hands-on experience using Azure’s data engineering stack.
- Should have implemented projects using programming languages such as Scala or Python.
- Working experience on SQL complex data merging techniques such as windowing functions etc..
- Hands-on experience with on-prem distribution tools such as Cloudera/Horton Works/MapR.
-
Data Engineer
12 hours ago
Vijayawada, India Tata Consultancy Services Full timeGreetings from TATA Consultancy ServicesJob Openings at TCSSkill :AZURE DATA ENGINEERExp range :4+10 YEARSRole : Permanent RoleJob location :HYDERABADCurrent location : HYDERABADMode of Interview: FACE TO FACE(WALKIN)INTERVIEWPls find the Job Description below.Keywords For SearchAzure Databricks, Python, ADF, Data Engineer, SynapseJob DescriptionRole and...
-
Data Engineer
2 days ago
Vijayawada, India Tata Consultancy Services Full timeGreetings from TATA Consultancy ServicesJob Openings at TCSSkill :DATABRICSExp range :5-8 YEARSRole : Permanent RoleJob location :HYDERABADCurrent location : HYDERABADInterview date :18TH JAN 25(SATURDAY) WALKIN INTERVIEWInterview mode : FACE TO FACE AT HYDERABAD (KOHINOOR PARK) Pls find the Job Description below. D· Minimum of 8 years of experience in data...
-
Data Engineer
3 weeks ago
Vijayawada, India Tata Consultancy Services Full timeGreetings from TATA Consultancy ServicesJob Openings at TCSSkill :Sr. Azure Data Engineer with data bricksExp range :5+ yearsRole : Permanent RoleJob location :bangaloreCurrent location : Anywhere In IndiaInterview mode : MS Teams Pls find the Job Description below. Job Description for Sr. Data Engineer Role –Data Engineer profile with 8+ years’...
-
AWS Data Engineer
3 weeks ago
Vijayawada, Andhra Pradesh, India Programmers Full timeAbout the Role:As an AWS Data Engineer in our team, you will play a crucial part in shaping the future of data-driven decision-making for our clients. We are seeking an experienced professional with a strong background in data engineering, analytics, and marketing to join our crew.Job Summary:We are looking for a talented AWS Data Engineer to lead our...
-
Senior Data Engineer
1 month ago
Vijayawada, India Spendflo Full timeOverview:As a Senior Data Engineer, you will be responsible for building and maintaining the entire data infrastructure at Spendflo. This includes designing the data models, creating and optimizing ETL pipelines, and ensuring the scalability, reliability, and performance of our data systems. You will work closely with cross-functional teams to enable...
-
AWS Data Engineer
4 months ago
Vijayawada, India FLYONIT Full timeJob Summary:We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing and...
-
Vijayawada, Andhra Pradesh, India Truelancer Full timeJob Title:AWS Cloud Data Engineer (Snowflake and AWS)About TruelancerWe are a leading global freelance marketplace that connects businesses with skilled professionals.Salary Range:$120,000 - $200,000 per yearCompany Overview:Truelancer is a technology company that empowers businesses to find the right talent for their projects. We believe in creating a...
-
Data Engineer
3 months ago
Vijayawada, India Material Full timeRole :- AWS Data EngineerLocation :- Gurgaon / BangaloreNotice :- Immediate Joiners Only Skills:- AWS , Spark , Python , Glue , S3 , Redshift etc.. Experience:- 4+ YearsPosition Overview: We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. The ideal candidate will have extensive experience with AWS Glue, Apache Airflow,...
-
Data Engineer
1 month ago
Vijayawada, India Wipro Full timeDear Candidates,Wipro is Hiring!!!Informatica Data CatalogExperience- 6+ YearsLocation- PAN IndiaNotice Period - Immediate - 30 daysRoles & Responsibilities:Overall experience of 6+ years with good experience of working in Informatica Data Catalogue Implementation. Proven experience with Informatica Data Catalog.Strong understanding of Data Discovery,...
-
Staffbee Solutions INC | Senior Data Engineer
4 weeks ago
Vijayawada, India Staffbee Solutions INC Full timeLooking for Freelance Opportunity In Data engineering?We are seeking theData Engineer Panelist @ 8+ yearsof ExperienceThis Is aFreelace_WFH/Remote_USA Shift OpportunityMust have atleast8+Years of exp at any one One cloud Tech@ GCP, AZure or AWSshould have excellent knowledge @Hadoop, python/pyspark, Bigdata,SQL & ETLRole Overview:We are seeking a highly...
-
Pentaho Data Integrator
7 days ago
Vijayawada, India Code Tree Software Solutions Full timeExperience Required - 3-5 Years Exp, 2-3 Years Exp Skills/Resources Required - 1 Senior, 1 Junior Minimum 3 years exp Pentaho, DWH, ETL, Data Model, RDBMS Systems, Oracle, Hadoop. As a data integration engineer you are responsible for the design, development, maintenance and documentation of our new Pentaho data integration (ETL) engine. You specialize in...
-
Informatica Data Quality
3 months ago
vijayawada, India RandomTrees Full timeImmediate #HIRING for a highly motivated and experienced Informatica Data Quality (IDQ) Developer to join our growing team.We’re a leading software company specializing in Artificial Intelligence, Machine Learning, Data Analytics, Innovative data solutions, Cloud-based technologies.If you're passionate about building robust applications and thrive in a...
-
Data Scientist Specialist
1 month ago
Vijayawada, Andhra Pradesh, India TechnoRight Full timeJob OverviewAt TechnoRight, we are seeking a highly skilled Data Scientist to join our team. This role is responsible for building and deploying machine learning models that drive business outcomes.About the RoleThis position requires a proven track record in research, data science, and engineering around building and shipping machine learning or statistical...
-
Lead Solutions Architect
2 months ago
Vijayawada, India Wipro Full timeHi Linkies,We are hiring for Lead Architect roleMandatory Skills: Cloud Native,GRPC,Microservices Architecture,Restful,GCP,Middleware TechnoologiesCertification: TOGAF or Cloud GCPWork Location: Bangalore/Hyderabad/ChennaiExperience: 13 Years to 17 YearsNotice Period: 30- days-60 daysAbility to understand & deep dive into product, architecture, engineering...
-
Business Data Analyst
3 weeks ago
Vijayawada, India NEXUS CONSULTING SOLUTIONS Full timeExperience: 8+ YearsSkills :1-3 years designing and delivering world-class software products in a Product Management role Domain knowledge in Data Privacy, Legal and Compliance—including technologies and experiences—as well as compliance, is highly desirable.Demonstrated curiosity and ability to learn multiple functional areas of a business is...
-
Vijayawada, Andhra Pradesh, India Turing Full timeAbout the JobTuring is seeking a seasoned Technical Lead to oversee and manage a team of skilled developers specializing in Lightning Web Components (LWC) and Apex development. This full-time role demands expertise in designing, developing, and maintaining complex solutions with a focus on high-quality output, efficiency, and scalability.Key...
-
Senior Engineer
3 months ago
vijayawada, India Velotio Technologies Full timeVelotio Technologies is a product engineering company working with innovative startups and enterprises. We have provided full-stack product development for 110+ startups across the globe, building products in the cloud-native, data engineering, B2B SaaS, IoT & Machine Learning space. Our team of 400+ elite software engineers solves hard technical problems...
-
Operations Lead
3 weeks ago
Vijayawada, India Esscents Full timeJob Title: Operation HeadCompany: EsscentsLocation: PuneEmployment Type: Full-timeAbout UsEsscents is a premium perfume brand inspired by luxurious scents, offering a curated collection of high-quality fragrances that evoke elegance and individuality. As a growing brand, we’re passionate about delivering exceptional experiences to our customers through...
-
DevOps Engineer
4 weeks ago
Vijayawada, India Cimba.ai Full timeCompany DescriptionCimba.AI is a Gen-AI-native platform that empowers enterprises to create custom adaptive AI agents. These agents can be built without any software engineering or coding knowledge, and they can generate deep insights and recommend business actions based on internal data. The platform utilizes self-training artificial intelligence to learn...
-
AdMax Local | Marketing Lead
2 weeks ago
Vijayawada, India AdMax Local Full timePosition:Marketing LeadLocation:Remote (India)Shift Timings:US Hours (PST)About Us:AdMax Local is a dynamic SaaS company revolutionizing marketing strategies for small and medium-sized businesses (SMBs) across the United States. Our platform, Amp'd Local, delivers personalized, data-driven recommendations, enabling businesses to optimize their marketing...