Big Data Engineer
4 weeks ago
Key Responsibilities:
- Develop and implement data ingestion pipelines and ETL processes using PySpark and Python capable of managing and processing large volumes of data, ranging in the millions of records.- Design and develop APIs using Python to facilitate data integration and processing.- Deploy, configure, and maintain Hadoop ecosystem components including HDFS, YARN, Hive, and HBase in on-premises environments.- Optimize the performance of Big Data applications and queries, ensuring efficient resource utilization.- Manage and monitor cluster health, including tuning and troubleshooting performance issues.- Ensure data security, integrity, and availability within the Big Data infrastructure.- Collaborate with cross-functional teams to integrate data analytics solutions and support business requirements.
Required Skills and Qualifications:
- Bachelor’s degree in computer science, Engineering, or a related field;- Proven experience 3+ years working with Big Data technologies such as Hadoop, PySpark, and MySQL in on-premises environments and handling large-scale data volumes, including datasets with millions of records, in on-premises environments.- Strong programming skills in Python, with experience in developing APIs.- Hands-on experience in Linux.- Expertise in deploying, managing, and optimizing Big Data clusters in a local data center.- Proficiency in monitoring and performance tuning of Hadoop and PySpark clusters.- Ability to troubleshoot and resolve complex issues related to data processing and infrastructure.- Excellent problem-solving skills and the ability to work independently as well as part of a team.- Strong communication skills with the ability to convey technical concepts to non-technical stakeholders.- Experience in dockers
Preferred Skills:
- Experience with Apache Airflow, Kafka, Apache Flink, and Apache NiFi for data orchestration and streaming.- Certification in Hadoop or related Big Data technologies.
Application Process: Interested candidates should submit their resume and a cover letter outlining their qualifications and experience in Big Data engineering and on-premises cluster management and include details of at least one relevant project involving the handling of large data volumes (in millions of records), you have worked on, including a brief description of the project, your role and the technologies used. This is mandatory requirement for our screening process.
-
Data Engineering Expert for Revenue Growth
2 weeks ago
Solapur, Maharashtra, India Cardinal Health Full timeAbout UsAt Cardinal Health, we are a leading distributor of pharmaceuticals and a global manufacturer and distributor of medical and laboratory products. We strive to be the most trusted partner in healthcare by delivering innovative digital solutions and end-to-end services that advance healthcare and improve lives.We have a long history of success, with...
-
Sr. Data Architect-Contract
5 days ago
Solapur, India Gravity Infosolutions Full timeRole: Sr. Data Architect-ContractExperience: 14-20 yearsShift: Regular IST shiftType: ContractDuration: 6 monthsLocation: Mumbai-Hybrid(2 Days Work from Office in a week)Job Description:We are seeking an accomplished Senior Data Architect to join our team and lead our data architecture efforts. As a Senior Data Architect, you will be responsible for...
-
Data Engineer
4 weeks ago
Solapur, India Truelancer.com Full timeJob Title: Data Engineer - DBT (Data Build Tool)Experience: 5-8 yrsMust-Have Skills: Airflow, S3, Snowflake, SQL/No SQL, Talend, AWSEssential Skills, Education and Experience:Should have a B.E. / B.Tech. / MCA or equivalent degree along with 4-7 years of experience in Data EngineeringStrong experience in DBT concepts like Model building and configurations,...
-
Data Engineer
4 weeks ago
Solapur, India Bonzer Business Solutions Pvt. Ltd. Full timeAs part of the team, you will be responsible for building and running the data pipelines and services that are required to support business functions/reports/dashboard.. We are heavily dependent on BigQuery/Snowflake, Airflow, Stitch/Fivetran, dbt , Tableau/Looker for our business intelligence and embrace AWS with some GCP.As a Data Engineer you’ll...
-
Data Engineering Lead
6 days ago
Solapur, Maharashtra, India Azilen Technologies Full timeCompany OverviewAzilen Technologies is a leading technology firm that specializes in data engineering and analytics solutions.Job SummaryWe are seeking an experienced Data Engineering Lead to join our team. As a key member of our data engineering group, you will be responsible for designing and developing scalable data pipelines, ensuring high-quality data...
-
Senior Data Engineer
4 weeks ago
Solapur, India GS Lab Full timeExperience Range : 7- 12 Years Responsibilities:Design, implement and maintain scalable ETL jobs and data pipelines for processing and analyzing large data setsBuild warehouses, datalakes and integrations with CRMs and analytics tools.Build and maintain data pipelines using Python for data extraction, transformation and loading (ETL) workflows.Ensure high...
-
Data Analytics Engineer
5 days ago
Solapur, India MPC Cloud Consulting Pvt Ltd Full timeWe are looking for Data Analytics Engineers Key Details:Role: Data Analytics Engineer (5-8 years of experience, ability to work independently) Skillset: Azure Data Factory, Azure Synapse Analytics, and other Azure services. SQL Server, Azure SQL Python software development stacks, ecosystems, frameworks, and tools such as NumPy, SciPy, Pandas, sci-kit-learn...
-
Data Science Engineer
7 days ago
Solapur, India RecourseTech Full timeRecourse Tech is a dynamic and fast-growing software and technology agency that builds cutting-edge solutions for businesses across industries. We specialize in delivering high-impact, data-driven solutions, leveraging advanced technologies, and empowering organizations with actionable insights.We are looking for a Data Science Engineer with 1-3 years of...
-
Principal Data Scientist
4 months ago
Solapur, India v4c.ai Full timeOverview: As the apex of the data science hierarchy, the Principal Data Scientist will set the strategic direction for data-driven initiatives, ensuring alignment with organizational goals, and lead in the adoption of advanced technologies including Machine Learning, Deep Learning, and Generative AI.Job Description:Strategically lead the planning and...
-
Solapur, Maharashtra, India Hexaware Technologies Full timeHexaware Technologies is a leading IT consulting and business solutions company that offers end-to-end services to clients worldwide. Our team of experts provides innovative solutions to businesses, helping them achieve their goals and stay ahead in the market.We are seeking an experienced Snowflake Data Engineer who will play a key role in designing,...
-
Senior Staff Software Engineer
3 months ago
solapur, India Coinbase Full timeAt Coinbase, our mission is to increase economic freedom around the world, and we couldn’t do this without hiring the best people. We’re a group of hard-working overachievers who are deeply focused on building the future of finance and Web3 for our users across the globe, whether they’re trading, storing, staking or using crypto. Know those people who...
-
GCP Data Engineer
4 weeks ago
Solapur, India True Tech Professionals Full timeNotice Period: Only 15 days or less will be consideredLocation: Remote (But will have to visit Gurgaon or Bangalore office – as and when required)Experience: 5 Plus YearsRoles/Responsibilities (Candidate’s responsibilities):Experience in GCP, Google BigQuery (Including BQ query standards, partitioning/clustering, data-model)Python Programming ExpertSQL...
-
Maltem Australia | Azure Data Engineer | solapur
4 weeks ago
solapur, India Maltem Australia Full timeAbout Maltem AustraliaMaltem Australia is a global boutique data and analytics consulting firm specialising in supporting organisations in designing and delivering enterprise data platforms and advanced analytics solutions. With strong expertise in building robust data architectures and leveraging cutting-edge technologies, we help businesses harness the...
-
Data Analyst
1 week ago
Solapur, India snooplay.in Full timeLocation: Noida (Full time)Education: B.Tech (preferred)About Snooplay:Snooplay is India’s leading marketplace for toys and games, where data drives every decision. With multiple data nodes fueling our platform, we rely on meaningful insights to improve experiences for parents and children. If you’re passionate about digging into data, deriving patterns,...
-
SAS Developer
1 week ago
Solapur, Maharashtra, India Cardinal Health Full timeWe are seeking a highly skilled SAS Revenue Management professional to join our Advanced Analytics and Automation Team at Cardinal Health. This is an exciting opportunity to leverage your expertise in data analytics and engineering to drive business innovation and success.About the RoleThis Data Engineer position is critical for the success of supporting our...
-
Data Science Strategist Lead
2 weeks ago
Solapur, Maharashtra, India v4c Full timeOverview:v4c is a leading innovator in the data science industry, shaping the future of technology and business.Salary: $250,000 - $350,000 per yearJob Description:Data Science ProjectsWe are seeking an exceptional Principal Data Scientist to lead strategic planning and execution of innovative data science projects, focusing on Machine Learning, Deep...
-
Lead DevOps Engineer
4 weeks ago
Solapur, India DDN Full timeThis is an incredible opportunity to be part of a company that has been at the forefront of AI and high-performance data storage innovation for over two decades. DataDirect Networks (DDN) is a global market leader renowned for powering many of the world's most demanding AI data centers, in industries ranging from life sciences and healthcare to financial...
-
solapur, India Cognitio Analytics Inc Full timeExperience:1 to 8 years - DA/ Sr. DA/ AM /ManagerMust Haves:Python/PySpark SQL Advanced ExcelComm Skills Data Analysis and interpretation Project Management Story Telling Interest and ability to get to grips with the underlying business problem and value drivers of a projectGood to have:FlaskTeam ManagementAny kind of health care background eg, -...
-
Site civil engineer
2 months ago
Solapur, India MIT Vishwaprayag University Full timeABOUT US MIT Vishwaprayag University MIT-VPU is built on the motto of seamlessly preparing the next generation for sustainable living with a holistic approach by integrating scientific, aesthetic, and ethical aspects into education across disciplines. “That is Education which gives Lasting Delight ”. ABOUT JOB Designation: Site...
-
Senior Salesforce Developer
2 months ago
Solapur, India Aezion, Inc Full timeSr. Salesforce Developer/LeadLocation: Bengaluru/Coimbatore/MysuruFull timeNote: Immediate Joiners PreferredResponsibilities:• Develop and implement Salesforce declarative solutions, customizations, applications, extensions and integrations to deliver solutions that impact multiple platforms and products• Perform code reviews, ad-hoc pair programming;...