Freelancer Big Data Engineer
2 days ago
There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you You will report into IIT'ans/BITS grads with 10+ years of development experience + work with F500 companies (our customers).
Company Background - We are a multinational software company that is growing at a fast pace. We have offices in Florida & New Delhi. Our clientele spreads across the US, Australia & APAC. To give you a sense of our growth rate, we've added 70+ employees in the last 6 weeks itself and expect another 125+ by the end of Q4 2024.
Key Responsibilities :
System Design and Development :
Architect, design, and implement significant components of our distributed data lake query engine.
Optimize for high scale and low latency, ensuring robust and efficient systems.
Write performant code tailored to modern hardware and leverage CPU capabilities effectively.
Cloud-Native Architectures :
Build and deploy systems leveraging containers (Kubernetes, Docker) and serverless technologies.
Work with object stores and cloud-native patterns to design scalable solutions.
Data Landscape Expertise :
Develop and optimize systems for distributed databases, OLAP engines, and file formats like Parquet.
Familiarity with tools like Spark, Trino, Databricks, and Snowflake.
Mentorship and Leadership :
Mentor junior engineers, fostering a culture of learning and innovation.
Contribute to high-level architectural decisions and long-term strategy.
Must-Have Skills :
Deep Understanding of Distributed Systems :
Expertise in architectural patterns (e.g., event-driven, microservices) and handling failures in distributed environments.
Familiarity with consensus algorithms and distributed system challenges.
Experience in designing and deploying production-grade distributed systems.
Cloud Expertise :
Hands-on experience with Kubernetes, containers, and serverless architectures.
Strong understanding of object storage systems and cloud-native design principles.
Data Engineering Foundations :
Knowledge of distributed databases (e.g., Cassandra), replication, and partitioning.
Familiarity with OLAP vs. OLTP workloads and columnar storage formats like Parquet.
Programming Proficiency :
Advanced skills in Java or equivalent languages.
Hands-on experience with multiple programming languages and frameworks.
Performance Optimization :
Strong understanding of modern hardware (CPUs) and techniques for writing low-level, performance-oriented code.
What we need
-~35 hours of work per week.
-100% remote from our side
-You will be paid out every month.
-Min 3 yrs of experience
-Please apply only if you have a 100% remote job currently
-If you do well, this will continue for a long time :)
-
leading MNC | Freelancer Big Data Engineer
24 hours ago
Delhi, India leading MNC Full timeLooking for aFreelancer Big Data Engineer| Remoteto join a team of rockstar developers. The candidate should have a minimum of 12+ yrs. of total experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You...
-
Big Data Engineer
2 months ago
Delhi, India Pert Telecom Solutions Full timeJob Description : We are seeking a skilled Big Data Engineer with deep expertise in PySpark, Python, MySQL, Hadoop and Linux. The ideal candidate should have extensive experience deploying and managing these tools within on-premises environments (no public cloud). Proficiency in cluster management, performance optimization, and monitoring is essential for...
-
Big Data Engineer
2 months ago
delhi, India Pert Telecom Solutions Full timeJob Description : We are seeking a skilled Big Data Engineer with deep expertise in PySpark, Python, MySQL, Hadoop and Linux. The ideal candidate should have extensive experience deploying and managing these tools within on-premises environments (no public cloud). Proficiency in cluster management, performance optimization, and monitoring is essential for...
-
Big data engineer
2 months ago
Delhi, India Pert Telecom Solutions Full timeJob Description : We are seeking a skilled Big Data Engineer with deep expertise in Py Spark, Python, My SQL, Hadoop and Linux. The ideal candidate should have extensive experience deploying and managing these tools within on-premises environments (no public cloud). Proficiency in cluster management, performance optimization, and monitoring is essential...
-
Big Data Engineer
2 months ago
Delhi, India Pert Telecom Solutions Full timeJob Description:We are seeking a skilled Big Data Engineer with deep expertise in PySpark, Python, MySQL, Hadoop and Linux. The ideal candidate should have extensive experience deploying and managing these tools within on-premises environments (no public cloud). Proficiency in cluster management, performance optimization, and monitoring is essential for this...
-
Big Data Engineering Manager
3 weeks ago
Delhi, Delhi, India NucleusTeq Consulting Pvt. Ltd. Full timeAbout NucleusTeq Consulting Pvt. Ltd.NucleusTeq Consulting Pvt. Ltd. is a technology consulting firm that helps businesses leverage big data and analytics to drive business outcomes.Salary and Benefits:The estimated salary for this position is around $120,000 - $180,000 per annum, depending on experience. We also offer a comprehensive benefits package,...
-
Big Data engineer
4 weeks ago
Delhi, India Tata Consultancy Services Full timeRole ; Big Data engineer Location : Pune, Chennai, MumbaiExp : 3-6 YearRequired Technical Skill Set** Hadoop, Python, PySpark, HIVEMust-Have**· Hands-on experience of Hadoop, Python, PySpark, Hive, Big Data Eco System Tools. · Should be able to develop, tweak queries and work on performance enhancement. · Solid understanding of object-oriented...
-
Big Data Automation Engineer
4 weeks ago
Delhi, Delhi, India PubMatic Full timeJob DescriptionWe are seeking a highly skilled Big Data Automation Engineer to join our team at PubMatic. As a key member of our engineering team, you will be responsible for designing, developing, and maintaining automation frameworks for big data workflows.Responsibilities:Designing and implementing automation frameworks using Python to automate big data...
-
Big Data Engineer
1 month ago
Delhi, Delhi, India Wipro Full timeWe are seeking a highly skilled Big Data Engineer to join our team at Wipro. As a Big Data Engineer, you will play a key role in designing, developing, and maintaining large-scale data processing systems.The ideal candidate will have a strong background in Hadoop and related technologies such as Scala. You will work closely with cross-functional teams to...
-
Lead Big Data Engineer
3 weeks ago
Delhi, Delhi, India Kloud9 Full timeAbout the JobKloud9 is seeking a highly skilled Lead Big Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for leading the design and implementation of large-scale data engineering projects on the Google Cloud Platform (GCP).The ideal candidate will have extensive experience in building data ingestion...
-
Hcltech | big data engineer
3 weeks ago
Delhi, India HCLTech Full timeWe are seeking a talented Scala and Spark Engineer to join our team. The ideal candidate will have a strong background in Scala and Spark development, with experience in building scalable and reliable data processing pipelines. Additionally, familiarity with Google Big Query for backend connectivity is highly desirable.Responsibilities:Develop and maintain...
-
Data Engineer for Big Data Solutions
4 weeks ago
Delhi, Delhi, India LTIMindtree Full timeWe are seeking an experienced Data Engineer to join our team at LTIMindtree. As a key member of our big data solutions group, you will be responsible for designing and developing scalable data platforms using modern technologies like Spark, HBase, Hive, and other Hadoop ecosystem tools.About the RoleIn this role, you will work closely with cross-functional...
-
Big Data Engineering Manager
3 weeks ago
Delhi, Delhi, India Strategic HR Solutions Full timeJob SummaryWe are looking for an experienced Data Engineer to join our team in Dubai. The successful candidate will be responsible for designing, building, and maintaining large-scale data processing systems using Informatica Big Data Management.ResponsibilitiesDesign and build data pipelines using HQL, Hive, and HDFSMaintain and optimize existing systems to...
-
Big Data Developer
4 weeks ago
Delhi, India Impetus Full timeLead Software EngineerLocations- Indore, Noida, HyderabadJob DescriptionWe are looking for a Lead Big Data Developer to design and oversee the implementation of large-scale data solutions, lead a talented team, and ensure the efficient processing and analysis of massive datasets.Roles & ResponsibilitiesKey Responsibilities- Architect Data Solutions: Design...
-
Big Data Engineer at Coffeee
3 weeks ago
Delhi, Delhi, India Coffeee Full timeCoffeee is seeking a skilled Hadoop Developer to join our team in Chennai/Bangalore. As a key member of our big data engineering team, you will be responsible for designing, implementing, and supporting Hadoop-based solutions. This role requires strong technical expertise in the Hadoop ecosystem, proficiency in Spark and Scala, and familiarity with...
-
Data Engineering Lead with Expertise in Big Data
3 weeks ago
Delhi, Delhi, India Aidetic Full timeAidetic seeks a seasoned Data Engineering Lead to oversee the design and deployment of scalable data architectures and ETL pipelines. As a key member of our team, you will lead a group of data engineers in building, maintaining, and optimizing complex data solutions for real-world projects.With 4+ years of experience in big data technologies, including...
-
Big Data Architect
2 weeks ago
Delhi, Delhi, India QAAgility Technologies Full timeWe are seeking a highly skilled Big Data Architect to join our team at QAAgility Technologies. As a key member of our big data team, you will be responsible for designing, building, and maintaining large-scale big data architectures on AWS.The ideal candidate will have 6+ years of experience in AWS, Big Data, and Python, with a strong background in data...
-
Cloud Data Engineer
4 weeks ago
Delhi, Delhi, India Tata Consultancy Services Full timeJob Role: Cloud Data Engineer - Big Data Solutions SpecialistAbout the Role: Tata Consultancy Services is seeking a skilled Cloud Data Engineer to join our team in Chennai. As a key member of our big data solutions team, you will be responsible for developing and deploying scalable big data pipelines on Google Cloud Platform (GCP).Responsibilities: Develop...
-
Data Engineering Expert for Big Data Systems
4 weeks ago
Delhi, Delhi, India NEXUS CONSULTING SOLUTIONS Full timeWe are seeking a skilled Data Engineer to join our team at NEXUS CONSULTING SOLUTIONS. This role requires experience in working with large data sets and a strong understanding of big data systems.Job DescriptionA successful candidate will have hands-on experience in Python and Scala programming languages. Additionally, they should be familiar with the Spark...
-
Architect : Big Data
7 months ago
Delhi, India Databricks Full timeAs a **Solutions Architect** (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a **technical advisor **internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve...