• Full-time

Publication date: 2024.10.07

Company name not disclosed

[AI Information Analysis] Data platform engineer (12572)

  • R&D / Engineer /Technician (AI, Data)
  • Tokyo
  • Salary ¥5.8M~7.3M
[AI 情報分析] データプラットフォームエンジニア (12572)
職種Position R&D / Engineer /Technician (AI, Data)
会社概要Company profile This position is available only for people residing in Japan.

The company recently raised 4.5 billion yen, bringing its total funding to 8.8 billion yen, and provides products that utilize AI.

Some of their products include a service that uses AI to analyze 30,000 domestic and international media outlets and delivers news that will interest users, such as related information, new fields, macro information, and overseas information, as well as a service that uses AI to analyze market trends and competitor movements to support strategy formulation.

Their mission is to support the digital transformation of all companies and organizations through products equipped with AI.
AI structures text data that was previously difficult to analyze and converts it into valuable data. By utilizing text data both inside and outside the company, they support the transformation into an organization that truly faces customers.
仕事内容Job description [Background of recruitment]
・ Applications and infrastructure development and operation of clawing/content extraction/distribution of several hundreds of millions of web documents are required.
・ It is necessary to develop and operate data pipelines to analyze and distribute the literature on the web (news, papers, patents) and customer literature.
・ It is necessary to build a monitoring base for stably collecting and expanding the literature on the web.

[Attractive Points of Company/Work]
By engaging in the development and operation of all products common document distribution systems, you can gain the following experiences.
・ You can be involved in serverless dispersion processing of several hundreds of millions of document data.
・ By engaging in the development of web crawlers for tens of thousands of sites, you can get the know -how of unstable system monitoring and operation design while utilizing both the knowledge of both web front and backends.
・ You can experience commitment to business by proposing and developing new development and improvement measures while communicating with the PDM and Customer Success team.

[Job Description]
They will design, develop, and operate the web crawler and document distribution pipeline located in the data distribution system common to all products.
They hope that the data platform team will lead to maximize productivity while developing the development to solve the issues as an engineer.

・ Development and operation of data platforms that distribute web news/patent/papers using TypeScript/python
・ Create a Japanese dataset required for learning a large -scale language model (LLM)
・ New construction and operation of analysis logic and data pipeline to analyze/distribute customer -in -house literature
・ Improvement and monitoring design of the scalability of distributed processing by FaaS/CaaS
応募資格Requirement [Requirements]*All required
・ Application development experience using python or typeScript (node.js)
・ Experience from requirements to design -implementation -operation.
・At least one of the following experiences
-The full stack development/operation using serverless and its lead experience (Lambda, ECS, Fargate, Step Functions, Cloud Functions, Cloud Run, etc.)
-DataPipeline-ETL construction, workflow engine, distributed processing framework development/operation experience (Hadoop, Spark, etc.)
Japanese language ability: Able to interview in Japanese.

[Preferred Requirements]
・ Master or doctoral in the technical field related to computer science
・ Experience of leading team development
・ Development and operation experience of their own development service for customers
・ Development and operation experience of services that perform document search and document recommendation
・ Wide knowledge and business experience for serverless
・ Development/operation experience using TERRAFORM
・Business-level English skills
日本語力Japanese level Advance (Able to interview)
雇用形態Employment type Full-time
勤務エリアLocation Tokyo
勤務時間Working hours Flexible (core time 10:00 to 14:00)
想定年収Salary ¥5.8M~7.3M
条件・待遇Condition Employment Type: Permanent
Trial period: 3 months
Estimated annual income: ¥5.8M~7.3M
Work location: Tokyo (work in the office or remotely depending on the job)
Working hours: Flexible (core time 10:00 to 14:00)
Holidays: 2 days a week (Saturdays and Sundays), national holidays
Vacation: Summer vacation, year-end and New Year holidays, and other vacations available
Overtime: 10~20 hours
Benefits: Rent allowance, travel expenses upon joining the company, overseas medical insurance, etc.
選考についてProcess Number of applicants: 1
Documents required for application: Work history, resume
Online interview: Available
Interview language: Japanese
Number of interviews (flow): Document screening → interviews(2~3 times)
Application for overseas residents: Unavailable
* May change depending on the situation of the candidate
Comments from United World
TOP