Python Data Engineer

2 months ago


solapur, India CredHive Full time

Position Overview

We are a seed-funded startup focused on using state-of-the-art AI technologies to revolutionize the credit industry. Our team consists of experts in machine learning and software engineers who have worked at top-tier US tech companies like Apple, Amazon, etc , and we are passionate about using AI to improve access to credit information for businesses. We have the product on the market, the first clients, and sufficient runway. We are seeking a highly skilled and innovative Python Data Engineer with a focus on parsing and scraping complex data from multiple sources . As a foundational member of our engineering team, you will play a crucial role in establishing and shaping our data team, focusing on extracting and managing large volumes of data essential for our services.


Key Responsibilities

  • Web Scraping and Data Collection: Design, build, and maintain robust web scraping systems to collect data from various online sources.
  • Strong Python: Ability to write clean and efficient Python while adhering to OOP principles. Able to parse and understand different kinds of data like XML, etc.
  • AWS Data Management: Set up and manage our data infrastructure on AWS, including services like S3, RDS and EC2.
  • Quality Assurance: Implement systems to ensure the accuracy and integrity of data, including validation and cleansing processes.
  • Innovative Problem-Solving: Stay ahead of the curve with emerging technologies and methodologies in data engineering and web scraping.


Qualifications

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
  • Strong experience in Python and web scraping tools and frameworks (such as BeautifulSoup, Scrapy,).
  • Basic experience in managing and scaling data on AWS.
  • Being scrappy
  • Being interested in Data and solving problems to gather any kind of data useful for our services.


What We Offer

  • Significant impact on the company’s data strategy and infrastructure.
  • A collaborative, flexible, and dynamic work environment.
  • Competitive salary and benefits package.

  • Data Engineer

    4 weeks ago


    solapur, India Fluidata Analytics Full time

    Role Name – Data Engineer (Python) Location- Remote Experience – 2-4 years Notice period – Immediate to 15 days (max) About Fluidata - Fluidata Analytics is a modern data and analytics consultancy firm. Our team of experts provide end-to-end data engineering and analytics solutions to clients by using cutting-edge data technology and...

  • Data Engineer

    1 month ago


    Solapur, India Deloitte Full time

    Hi, We are looking for Candidates who are having experience as Data Engineer Role.Experience Required: 7+ years.Location: PAN IndiaResponsibilities:Databricks Development: Design, develop, and optimize Databricks workflows and solutions.Python & PySpark: Write, test, and maintain Python and PySpark code for data processing and analytics.SQL Proficiency:...

  • Data Engineer

    4 weeks ago


    Solapur, India Truelancer.com Full time

    Job Title: Data EngineerExperience: 4+ YearsMandatory Skills: Azure data factory, python, apache-spark, Datalakehouse, Databricks, Should be able to create pipelines and transform dataKey Responsibilities:Design, develop, and maintain end-to-end data pipelines using Azure Data Factory, ensuring the ingestion, transformation, and integration of large...

  • Senior Data Engineer

    1 month ago


    solapur, India Info Services Full time

    Urgent Need of below requirementData Engineer - Offshore Pointers from managers meeting today Must have Skill - Experience with Operating Airflow - Understand how the whole cycle works and have worked on operating it. Many people know airflow, but not everyone knows the operating part. So this person must know that. Work revolves around Airflow in this team....

  • GCP Data Engineer

    5 days ago


    Solapur, India True Tech Professionals Full time

    Preference would be given to Engineers who can work from the office in a hybrid role in Gurgaon. Roles/Responsibilities (Candidate’s responsibilities):Experience in Google BigQuery (Including BQ query standards, partitioning/clustering, data-model)Python Programming ExpertSQL knowledgePreferred: Experience in Ascend toolCreate Data flow, Data service, Read...

  • Test Data Engineer

    1 month ago


    solapur, India EVERSANA Full time

    Experience: 8-10+yearsPrimary Skill: Leadership role, Hands on SQL Data Base. Data mining and data reporting testers Tester| ETL Analyst, ETL Tester.Responsibilities:This individual contributor is responsible for guiding and influencing the development of a formal Data Quality Services Framework and for supporting the development of software quality...


  • Solapur, India BeamX TechLabs Full time

    Job Title: Senior Python Developer Location: HyderabadExperience: 2-3 YearsEmployment Type: Full-TimeWe are looking for an experienced Senior Python Developer with a strong background in FastAPI to join our dynamic development team. You will work on building high-performance web applications and APIs while collaborating closely with cross-functional...


  • Solapur, Maharashtra, India Technogen, Inc. Full time

    Role: Sr Mgr-Data EngineeringExperience: 13-17 YearsLocation: BangaloreKey Skills: GCP, Big Query, Spark/Kafka/Pyspark/Scala, SQL, People managementAbout the Role:Lead the development and implementation of data engineering solutions using Google Cloud Platform and BigQuery.Design and implement data pipelines and workflow management systems.Collaborate with...

  • Sr. Data engineer

    3 weeks ago


    solapur, India Info Services Full time

    Hello Friends,,Hope everyone doing good!!We have very good opportunity with our direct client. We have a good hold on this position.This is a offshore opportunity and Remote.Below is the JD.SkillsetsStrong coding skills - Scala(preferred) or Python or Java : mandatoryStrong SQL skills : mandatorySpark or other similar big data framework experience (Flink,...

  • Data Engineer

    7 days ago


    Solapur, India Truelancer.com Full time

    Job Title: Data EngineerMandatory Skills: Pyspark, Azure, ADF, Databricks, ETL, SQLJDDesign and build data pipelines using Spark-SQL and PySpark in Azure DatabricksDesign and build ETL pipelines using ADFBuild and maintain a Lakehouse architecture in ADLS / Databricks.Perform data preparation tasks including data cleaning, normalization, deduplication, type...

  • Data Engineer

    3 weeks ago


    Solapur, India Truelancer.com Full time

    Job Title: Data EngineerMandatory Skills: Pyspark, Azure, ADF, Databricks, ETL, SQLJob Description:Design and build data pipelines using Spark-SQL and PySpark in Azure DatabricksDesign and build ETL pipelines using ADFBuild and maintain a Lakehouse architecture in ADLS / Databricks.Perform data preparation tasks including data cleaning, normalization,...


  • Solapur, India Deloitte Full time

    We are looking for Candidates who are having experience as AI/ML and Gen AI Engineer.Experience Required: 10+ years.Location: PAN IndiaResponsibilities:10+ years of experience in AI-ML/Gen AI (Data Science), with a focus on generative models, including but not limited to GANs (Generative Adversarial Networks), VAEs (Variational Autoencoders), and...

  • Data Analyst

    1 month ago


    Solapur, India fascinatid Full time

    Data AnalystRemote, FreelancerThis is an excellent opportunity to join our B2B content marketing agency. You will be part of an innovative team and report to the SEO Manager. (This is a Contract Position, there will be no benefits such as insurance, or any allowances other than the monthly payment)Role DescriptionThis is a part-time remote role for a Data...

  • Full Stack Engineer

    1 month ago


    Solapur, India SparkCognition Full time

    Position Summary:SparkCognition’s Renewable Suite is the world's leading AI enabled SaaS platform that provides asset performance management and prescriptive analytics software to ever growing renewable energy power plants by combining the power of machine learning, big data, and deep domain expertise. Our platform works across multiple asset types such as...


  • Solapur, India VNB Consulting Services Full time

    Microsoft Fabric Analytics EngineerLocation: RemoteExperience: 3-5 yearsAbout Us:VNB Consulting Services, headquartered in New Jersey, is an end-to-end technology solution provider specializing in Apps, Automation, Analytics, and AI. Our mission is to deliver tested technology solutions that enhance existing technology, improve business processes, or develop...

  • Senior Data Engineer

    1 month ago


    Solapur, India Visa Full time

    Job Description and ResponsibilitiesTeam SummaryVisa Consulting & Analytics (VCA) team is a key part of the Global Solutions organization, a high-performing team of data scientists, data analysts and statisticians helping major organizations adapt and evolve to meet the changes taking place in technology, finance, and commerce, with cutting-edge, creative...

  • Full Stack Engineer

    1 month ago


    Solapur, India White Force Group Full time

    3+ years of professional experience in Python· Strong SQL expertise with hands-on experience in database design, optimization, and management.· 2+ years of experience with Nest JS or other backend frameworks.· Familiarity with AWS/GCP is a plus.· Ability to work independently as well as collaboratively within a team.· Strong problem-solving skills with...


  • Solapur, India Stealth Full time

    About UsWe are an innovative AI startup dedicated to revolutionizing industries with cutting-edge generative AI solutions. Our mission is to leverage advanced AI technologies to create impactful products that solve real-world problems. As we continue to grow, we are looking for a skilled Backend Engineer to join our team and help build the infrastructure...


  • solapur, India ThreatXIntel Full time

    Company DescriptionThreatXIntel is a startup cyber security company that offers customized, affordable solutions to protect businesses from cyber threats. Specializing in cloud security, web and mobile security testing, and DevSecOps, we aim to provide high-quality services to businesses of all sizes. Our proactive approach involves continuous monitoring and...


  • Solapur, India Canvendor Full time

    Greeting from Canvendor!We are hiring for Design verification Engineer at location Bangalore /Chennai/ NoidaRole: Design Verification EngineerLocation: Bangalore/Chennai/NoidaExp: 5-13yrsKey Responsibilities:IP verification Using SV/UVM or SOC Verification using C/SV Interconnect Protocols: AHB, AXI, APBSOC Interfaces: GPIO, SPI, I2C, UART (3+)High Speed...