Data engineer

2 months ago

Kolkata, India Webologix Ltd INC Full time
Job Position: AWS Data Engineer
Job Type: Full-Time
Location: Kolkata, IN
Experience: - 4+ to 10+ Years
Must Have: -Amazon S3, Lambda, Glue, AWS Step Function, RDS, Dynamo DB, Python, Py Spark, SQL, PL/SQL.
Key Responsibilities:
Data Pipeline Development: Design and implement robust data pipelines using AWS services such as S3, Glue, Lambda, and Step Functions to facilitate ETL (Extract, Transform, Load) processes and ensure data integrity and accessibility.
Data Storage Management: Utilize Amazon S3 for scalable and secure data storage. Optimize data storage solutions and manage data lifecycle policies to ensure efficient data retrieval and cost management.
Data Transformation and Processing: Develop and execute data transformation workflows using AWS Glue and Py Spark. Ensure data is processed efficiently and accurately to meet business requirements.
Database Management: Manage and optimize relational and No SQL databases using Amazon RDS and Dynamo DB. Implement data models and design schemas to support business applications and reporting needs.
Scripting and Automation: Write Python scripts to automate data processing tasks, integrate data from various sources, and develop custom solutions for data manipulation and analysis.
SQL and PL/SQL Development: Utilize SQL for querying relational databases and performing complex data manipulations. Apply PL/SQL for advanced database procedures and optimizations in Oracle environments.
Data Quality and Performance: Monitor and maintain data quality, performance, and security. Implement best practices for data management and troubleshoot issues to ensure reliable data operations.
Collaboration and Documentation: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and provide technical expertise. Document data pipelines, processes, and architecture to support ongoing maintenance and scalability.
Required Skills and Qualifications:
Proven experience as a Data Engineer or similar role with a focus on AWS technologies.
Strong proficiency in AWS services including S3, Lambda, Glue, Step Functions, RDS, and Dynamo DB.
Expertise in Python and Py Spark for data processing and automation tasks.
Solid experience with SQL and PL/SQL for querying and managing relational databases.
Demonstrated ability to design and implement data pipelines and workflows.
Strong problem-solving skills and attention to detail.
Excellent communication and collaboration abilities.
Qualifications:
Bachelor’s degree in computer science, Information Technology, or a related field.

  • Data engineer

    2 months ago

    Kolkata, India Webologix Ltd INC Full time

    Job Position: Azure Cloud DB EngineerJob Type: Full-TimeLocation: Kolkata, INExperience: - 3 to 8+ YearsMust Have: - Azure DATA Factory; Azure SQL; Azure Databricks; Azure Cosmos DB; Py Spark; PythonResponsibilities: Data Pipeline Development:Design and implement data integration workflows using Azure Data Factory to ingest, transform, and load data from...

  • Data Engineer

    4 weeks ago

    Kolkata, India s seth's assignment's Full time

    Job Title: Data EngineerLocation: Remote Contract Terms: 3 Months with the potential to extend for more than a yearOverviewWe are looking for a highly skilled and motivated Data Engineer with 4-8 years of experience to join our dynamic team. This role is ideal for individuals with advanced technical expertise in Python programming and relational databases,...

  • Data Engineer

    4 weeks ago

    Kolkata, India Gemraj Technologies Ltd Full time

    Data EngineerPosition: FTE, remote, IndiaAvailability-Immediate-1 weekOverlap: 4 hours with PST(Pacific Standard Time)Candidate proficiency level: Senior(8+)Responsibilities:Design, build and maintain core data infrastructure pieces that allow Aircall to support our many data use cases.Enhance the data stack, lineage monitoring and alerting to prevent...

  • Data Engineer

    4 months ago

    Kolkata, India Webologix Ltd INC Full time

    Job Position: Azure Cloud DB EngineerJob Type: Full-TimeLocation: Kolkata, INExperience: - 3 to 8+ YearsMust Have: - Azure DATA Factory; Azure SQL; Azure Databricks; Azure Cosmos DB; PySpark; PythonResponsibilities: Data Pipeline Development:Design and implement data integration workflows using Azure Data Factory to ingest, transform, and load data from...

  • Data Engineer

    3 months ago

    kolkata, India Webologix Ltd INC Full time

    Job Position: Azure Cloud DB EngineerJob Type: Full-TimeLocation: Kolkata, INExperience: - 3 to 8+ YearsMust Have: - Azure DATA Factory; Azure SQL; Azure Databricks; Azure Cosmos DB; PySpark; PythonResponsibilities: Data Pipeline Development:Design and implement data integration workflows using Azure Data Factory to ingest, transform, and load data from...

  • Data Engineer

    4 months ago

    Kolkata, India Webologix Ltd INC Full time

    Job Position: Azure Cloud DB EngineerJob Type: Full-TimeLocation: Kolkata, INExperience: - 3 to 8+ YearsMust Have: - Azure DATA Factory; Azure SQL; Azure Databricks; Azure Cosmos DB; PySpark; PythonResponsibilities: Data Pipeline Development:Design and implement data integration workflows using Azure Data Factory to ingest, transform, and load data from...

  • Data Engineer

    4 months ago

    Kolkata, India Webologix Ltd INC Full time

    Job Position: Azure Cloud DB Engineer Job Type: Full-Time Location: Kolkata, IN Experience: - 3 to 8+ Years Must Have: - Azure DATA Factory; Azure SQL; Azure Databricks; Azure Cosmos DB; PySpark; Python Responsibilities:  Data Pipeline Development: Design and implement data integration workflows using Azure Data Factory to ingest, transform, and...

  • Data Engineer

    1 month ago

    Kolkata, India Webologix Ltd INC Full time

    Job Position: AWS Cloud DB Engineer Job Type: Full-TimeLocation: Kolkata, INExperience: - 3+ to 10+ YearsMust Have: - Aws Glue; Aws Lambda; AWS RDS; AWS S3; Dynamo Db; PySpark; Python; AWS Step FunctionResponsibilities: Data Pipeline Development:Design and implement data ingestion and transformation workflows using AWS Glue.Develop and manage ETL...

  • Data Engineer

    2 months ago

    Kolkata, India Webologix Ltd INC Full time

    Job Position: AWS Cloud DB EngineerJob Type: Full-TimeLocation: Kolkata, INExperience: - 3+ to 10+ YearsMust Have: - Aws Glue; Aws Lambda; AWS RDS; AWS S3; Dynamo Db; PySpark; Python; AWS Step FunctionResponsibilities: Data Pipeline Development:Design and implement data ingestion and transformation workflows using AWS Glue.Develop and manage ETL...

  • Data engineer

    2 months ago

    Kolkata, India Webologix Ltd INC Full time

    Job Position: AWS Cloud DB EngineerJob Type: Full-TimeLocation: Kolkata, INExperience: - 3+ to 10+ YearsMust Have: - Aws Glue; Aws Lambda; AWS RDS; AWS S3; Dynamo Db; Py Spark; Python; AWS Step FunctionResponsibilities: Data Pipeline Development:Design and implement data ingestion and transformation workflows using AWS Glue.Develop and manage ETL...

  • Data Engineer

    1 month ago

    Kolkata, West Bengal, India FinArb Consulting Full time

    Job TitleSenior Data EngineerAbout the RoleWe are seeking a highly skilled Senior Data Engineer to join our team at FinArb Consulting. This is an exciting opportunity to work on large-scale data engineering projects, leveraging modern technologies to drive impactful insights.Key ResponsibilitiesDesign and build scalable data pipelines and workflows using...

  • Senior Data Engineer

    4 weeks ago

    Kolkata, India eInfochips (An Arrow Company) Full time

    Role: Senior Data EngineerYears of Experience: 5-10Location: Ahmedabad Overview:We are seeking a highly skilled and experienced Senior Software Engineer to join our Data Engineering team. The ideal candidate will have 5 to 10 years of experience in software engineering, with a strong focus on data engineering practices and an awareness of AI/ML and...

  • Senior Data Engineer

    2 months ago

    kolkata, India FinArb Consulting Full time

    Job Title: Senior Data EngineerResponsibilities:We are seeking a self-driven Senior Data Engineer to join our dynamic team of data and software engineers. Your primary responsibilities will include:Designing, building, and maintaining scalable data pipelines and workflows.Developing and optimizing data ingestion, transformation, and integration...

  • Senior Data Engineer

    2 months ago

    Kolkata, India FinArb Consulting Full time

    Job Title: Senior Data EngineerResponsibilities:We are seeking a self-driven Senior Data Engineer to join our dynamic team of data and software engineers. Your primary responsibilities will include:Designing, building, and maintaining scalable data pipelines and workflows.Developing and optimizing data ingestion, transformation, and integration...

  • Azure Data Engineer

    4 months ago

    Kolkata, India Talpro Full time

    Job Description : We are looking for a skilled and experienced Azure Data Engineer with expertise in ADF (Azure Data Factory), strong RDBMS and SQL skills, and familiarity with Informatica PWC and Oracle PL/SQL. As an Azure Data Engineer, you will be responsible for understanding and converting Informatica PWC and Oracle PL/SQL code into ADF and Azure SQL....

  • Data Engineer

    2 months ago

    Kolkata, India Webologix Ltd INC Full time

    Job Position: AWS Cloud DB EngineerJob Type: Full-TimeLocation: Kolkata, INExperience: - 3+ to 10+ YearsMust Have: -Aws Glue; Aws Lambda; AWS RDS; AWS S3; Dynamo Db; PySpark; Python; AWS Step FunctionResponsibilities: Data Pipeline Development:Design and implement data ingestion and transformation workflows using AWS Glue.Develop and manage ETL processes...

  • Data Engineer

    4 months ago

    Kolkata, India Webologix Ltd INC Full time

    Job Position: Azure Cloud DB EngineerJob Type: Full-TimeLocation: Kolkata, INExperience: - 3 to 8+ YearsMust Have: - Azure DATA Factory; Azure SQL; Azure Databricks; Azure Cosmos DB; PySpark; PythonResponsibilities:Data Pipeline Development:Design and implement data integration workflows using Azure Data Factory to ingest, transform, and load data from...

  • Data Engineering Specialist

    3 weeks ago

    Kolkata, West Bengal, India 2Coms Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at 2Coms. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining scalable and efficient data pipelines using tools like PySpark, Spark Scala, or Databricks.

  • Azure Data Engineer

    13 hours ago

    Kolkata, India Tata Consultancy Services Full time

    TCS is hiring for Azure Data EngineerExperience: 4+ YearsLocation: Kolkata, PAN IndiaMust-have skills: Azure DatabricksJob Description –Build and maintain the technology platform for both on premise and cloud-based solutionsResponsible for engineering a new cloud data platform analyzing different emerging technology solutions that range from Microsoft...

  • Gcp data engineer

    3 days ago

    Kolkata, India Global Infovision Private Limited Full time

    Job Title: GCP Data EngineerJob Location: Indore, Nagpur, Bhubaneswar & KolkataJob Type: Full-TimeClient: DirectExperience:7 years of experience in data engineering, with a focus on cloud-based solutions.Extensive experience with Google Cloud Platform (GCP) and its data services, including Big Query, Dataflow, Pub/Sub, Cloud Storage, and Cloud...