Freelance Cloud Data Engineer

3 weeks ago

delhi, India ThreatXIntel Full time
Company Description
ThreatXIntel is a startup cyber security company that offers customized, affordable solutions to protect businesses from cyber threats. Specializing in cloud security, web and mobile security testing, and DevSecOps, we aim to provide high-quality services to businesses of all sizes. Our proactive approach involves continuous monitoring and testing to identify vulnerabilities before they can be exploited.
Role Description
We are seeking an experienced  Cloud Data Engineer  with expertise in  AWS ,  Spark ,  Java , and  Ab Initio  graph transformation logic. This role involves working on data migration and transformation for a large-scale cloud data environment, with a focus on converting input data to output files based on transformation requirements. The ideal candidate will also have strong knowledge of  parallel and batch processing  and experience with  Yugabyte data loading .
Key Responsibilities:
Analyze  Ab Initio graph transformations  to understand input data structures, transformation logic, and output requirements.
Develop and optimize  Java Spark code  to transform  parquet files  and load data into  Yugabyte  and other storage formats.
Design and implement data migration and integration processes on  AWS EMR  using  Docker  and  Terraform  for deployment.
Collaborate with cross-functional teams to understand data requirements, perform data transformations, and troubleshoot issues.
Ensure high-quality code and efficient performance for data processing pipelines, especially in high-transaction environments.
Document processes, transformations, and code changes, providing technical support during the transition from legacy systems.
Qualifications:
5+ years of experience  in data engineering, with a focus on  AWS ,  Spark , and  Java  for cloud-based data migration.
Strong understanding of  Ab Initio graph transformations  and ability to interpret complex data flows and transformations.
Proficient in  AWS EMR  and data tools, with experience in  Docker  and  Terraform  for infrastructure as code (IaC).
Solid background in  parallel processing, batch processing , and handling large transaction volumes.
Knowledge of  parquet file formats  and loading data into  Yugabyte  or other distributed databases.
Experience with  CI/CD pipelines  and familiar with  Unix/Linux scripting .
Strong analytical skills and ability to independently troubleshoot and resolve complex data issues.
Preferred Skills:
AWS Certifications  and familiarity with  Eclipse IDE  and  Spring Boot  for Java applications.
Experience with orchestration tools like  Airflow  and containerization with  Kubernetes .
Why Join Us?
This is a high-impact role on a core data engineering team, offering the chance to work on cutting-edge cloud technologies and large-scale data migrations. If you have a passion for optimizing data transformations and a strong understanding of cloud-based data systems, we’d love to work with you

  • Freelance | Freelance: Looking for Site Reliability Engineering

    3 days ago

    delhi, India Freelance Full time

    This is the role for a Freelance: (IT - Cloud) Site Reliability Engineering (SRE) TrainerAs an SRE Trainer, you will be responsible for training and mentoring individuals in the field of Site Reliability Engineering, ensuring they have the skills and knowledge needed for success in the role.Duration: 5 WeekdaysQualificationsProven experience as a Site...

  • Freelance: Looking for Site Reliability Engineering

    2 days ago

    Delhi, India Freelance Full time

    This is the role for a Freelance: (IT - Cloud) Site Reliability Engineering (SRE) TrainerAs an SRE Trainer, you will be responsible for training and mentoring individuals in the field of Site Reliability Engineering, ensuring they have the skills and knowledge needed for success in the role.Duration: 5 WeekdaysQualificationsProven experience as a Site...

  • Freelance: looking for site reliability engineering

    22 hours ago

    Delhi, India Freelance Full time

    This is the role for a Freelance: (IT - Cloud) Site Reliability Engineering (SRE) TrainerAs an SRE Trainer, you will be responsible for training and mentoring individuals in the field of Site Reliability Engineering, ensuring they have the skills and knowledge needed for success in the role.Duration: 5 WeekdaysQualificationsProven experience as a Site...

  • Freelance: Looking for Site Reliability Engineering

    2 days ago

    Delhi, India Freelance Full time

    This is the role for a Freelance: (IT - Cloud) Site Reliability Engineering (SRE) TrainerAs an SRE Trainer, you will be responsible for training and mentoring individuals in the field of Site Reliability Engineering, ensuring they have the skills and knowledge needed for success in the role.Duration:5 WeekdaysQualificationsProven experience as a Site...

  • Senior Data Analyst

    1 month ago

    Bangalore/Hyderabad/Delhi/Kolkata, India Freelance Full time

    Key Responsibilities• Solve complex technical problems in a collaborative environment with minimal guidance and review• Implement diverse engineering tasks including gap, data, and impact analysis• Perform low-level design configuration, code customization, refactoring and review independentlyRequired Skills• TableauAbout the RoleThis is an exciting...

  • Site Reliability Engineering Trainer Opportunity

    1 day ago

    Delhi, Delhi, India Freelance Full time

    OverviewAs a Freelance Site Reliability Engineering Trainer, you will play a crucial role in empowering individuals with the skills and knowledge needed to succeed in this field.About the RoleThis is a remote freelance opportunity where you will be responsible for delivering training and mentoring programs to help learners acquire expertise in Site...

  • Freelance Graph Data Engineer

    2 days ago

    Delhi, India Leading MNC Full time

    Looking for aFreelance Graph Data Engineerto join a team of rockstar developers. The candidate should have a minimum of 5+ yrs. of experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will report...

  • Freelance Graph Data Engineer

    2 days ago

    Delhi, India Leading MNC Full time

    Looking for a Freelance Graph Data Engineer to join a team of rockstar developers. The candidate should have a minimum of 5+ yrs. of experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will report...

  • Freelance Graph Data Engineer

    2 days ago

    Delhi, India Leading MNC Full time

    Looking for a Freelance Graph Data Engineer to join a team of rockstar developers. The candidate should have a minimum of 5+ yrs. of experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will report...

  • Freelance graph data engineer

    22 hours ago

    Delhi, India Leading MNC Full time

    Looking for a Freelance Graph Data Engineer to join a team of rockstar developers. The candidate should have a minimum of 5+ yrs. of experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you! You will report...

  • Data Warehouse Engineer | Data Warehouse Engineer | delhi

    4 hours ago

    delhi, India Data Warehouse Engineer Full time

    Experience : 2- 5 YearsPrimary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills.Good to have skills: Power BI, Databricks, Python.

  • Senior Data Integration Engineer

    2 days ago

    Delhi, India Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement:Design and architect integration solutions to connect various enterprise applications, systems, and databases.Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.Utilize Azure Integration Services such as Azure Logic Apps,...

  • Senior Data Integration Engineer

    2 days ago

    Delhi, India Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement:- Design and architect integration solutions to connect various enterprise applications, systems, and databases.- Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.- Utilize Azure Integration Services such as Azure Logic...

  • Senior Data Integration Engineer | Senior Data Integration Engineer | delhi

    3 days ago

    delhi, India Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement:Design and architect integration solutions to connect various enterprise applications, systems, and databases.Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.Utilize Azure Integration Services such as Azure Logic Apps,...

  • Senior data integration engineer

    22 hours ago

    Delhi, India Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement:Design and architect integration solutions to connect various enterprise applications, systems, and databases.Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.Utilize Azure Integration Services such as Azure Logic Apps,...

  • Data Engineer

    4 weeks ago

    Delhi, Delhi, India Syren Cloud Inc Full time

    About the RoleWe are seeking a highly skilled Data Engineer - Cloud Computing Expert to join our team at Syren Cloud Inc. In this role, you will be responsible for designing, developing, and implementing distributed applications and systems on the Azure Cloud platform.Key ResponsibilitiesDevelop and implement databases, data collection systems, data...

  • Data Engineering Manager

    2 weeks ago

    Delhi, Delhi, India Cloud Destinations Full time

    Data Engineer Role at Cloud DestinationsWe are seeking a highly skilled Senior Data Engineer to play a critical role in designing, developing, and maintaining robust data pipelines and data warehouses at Cloud Destinations.Key Responsibilities:Data Engineering:Design, develop, and maintain efficient and scalable data pipelines using SQL, Python, and other...

  • Freelance webinar specialist

    1 month ago

    Delhi, India Softvolt - IT Solutions Full time

    This is a freelancing - hour based job roleCompany DescriptionSoftvolt is a Software Development and Consulting Services company specializing in Web Development, Mobile applications, DevOps, Integration, Business Intelligence, and Digital Marketing. We provide cost-effective and professional IT solutions to build lasting client relationships.Role...

  • Senior Data Engineer

    4 weeks ago

    Delhi, Delhi, India Cloud Destinations Full time

    Job Summary:Cloud Destinations is seeking a highly skilled Data Architect Leader to design, develop, and maintain robust data pipelines and data warehouses. As a Senior Data Engineer, you will play a critical role in extracting, transforming, and loading (ETL) large datasets, analyzing data to uncover insights, and building data products that drive business...

  • BI Tableau Engineer

    1 month ago

    Bangalore/Hyderabad/Delhi/Kolkata, IN Freelance Full time

    Job DescriptionSenior resource with minimum 2-8 Years of experience in Tableau.- Very strong experience on Tableau Functions, Tableau Data Prep and Tableau Visuals.- Good experience on end-to-end Enterprise BI development & deployment.- Developing data Models, Data Preparation with Data Flows and designing BI reports- Proven experience in writing advanced...