Data Engineer

Level

Middle

Department

Other IT Positions

Type

Full Time

Project

Kaltura

Locations:

Warsaw

Poland

Job Details

Posted on:

September 3, 2024

Job ID:

327

About the Company

Established in 2004, ALLSTARSIT was founded with a clear vision: to enhance the landscape of global IT employment by bridging the gap between companies and skilled professionals. The core belief was that assembling a team shouldn't be hindered by geographical constraints. Fast forward to the present day, ALLSTARSIT stands as an international outstaffing service provider committed to change the way businesses recruit, compensate, and oversee top talent worldwide.

With operational hubs scattered across Europe, Asia, and LATAM, and its headquarters situated in San Francisco, US, the company boasts a workforce of over 1,000 adept professionals. Spanning across more than 20 countries, ALLSTARSIT offers a diverse range of skilled employees across various verticals, including AI, cybersecurity, healthcare, fintech, telecom, media, and so on.

About the Project

Kaltura, recognized for its open-minded and innovative culture, is a leading company in the realm of video technology. We're committed to fostering a creative and dynamic environment where new ideas and cutting-edge solutions flourish. Our team is instrumental in shaping the future of video technology, driving change, and pushing boundaries in the digital landscape.

We're seeking a driven Data Engineer to join our dynamic team at Kaltura, spearheading our pioneering video platform as a service.

In this role, you will be part of the data engineering team. Your expertise will be crucial insteering the team's collaboration with various R&D teams to meet our product roadmap objectives.

‍

Specialization

Headquarters

New York, USA

Years on the market

Team size and structure

Current technology stack

Required skills:

Minimum of 2 years, with expertise in Python, Scala, or Java.
In-depth knowledge of Big Data technologies like Spark, Kafka, Redshift, Athena, S3, and Kinesis
Extensive experience with Amazon Web Services, including hands-on experience with Apache Airflow.
Proven ability to excel in ambiguous situations and create innovative solutions.

Scope of work:

Development and refinement of ETL infrastructure using Python/Scala/Java, SQL, and advanced AWS technologies like EMR and Redshift.
Drive innovation in our big data platform to efficiently process millions of events daily.
Architect and manage robust, scalable data pipelines that support rapid data growth.
Design and maintain ETL pipelines with enhanced monitoring and alert systems.
Mentor team members on optimizing data models for clarity and accessibility.
Oversee the management of metadata, data catalogs, and documentation to aid business decisions.
Ensure team proficiency in SQL for effective data querying and manipulation.
Direct the use of Databricks for complex data engineering and analytical tasks.

These would also be nice:

Exceptional problem-solving, analytical, and communication skills, fostering strong team collaboration.
Committed to continuous learning and staying current with industry trends, data engineering best practices, security, and compliance.
Highly adaptable to evolving project requirements and priorities, ensuring flexibility in dynamic environments.