I have numerous responsibilities regarding the development and system architecture using a wide range of technologies at Yahoo Japan’s Technology Incubation.
My biggest strength is that I’m a fast learner and can easily use new technology. Even if I get assigned to a new project and have to use unfamiliar technology, I can take over the leadership of this project in less than one week.
I’m exceptionally proficient in Go, and in addition to possessing abundant experience with different programming languages, I also have a profound knowledge starting from infrastructures such as Kubernetes, OpenStack, to vector search engines like Vald.
Technology Incubation is the R&D department. We work closely with our services and provide new technology to services.
Currently, we work on video analysis & advertisement optimization by image analysis & shopping item recommendation by using image feature vector analysis.
Product Owner of Technology Area 2020/4~present
Responsible for managing the Technology Incubation team’s advanced technology areas. Mainly manages three products, vector search technology (Vald), anomaly detection for infrastructure, and in-house NewSQL areas.
Next Generation Vector Search Engine Platform Vald
Founder & Lead Developer (Go/C++/Python/Kubernetes/OpenStack) 2019/8~present
I’m working on the development of distributed approximation KNN graph and tree dense-vector search engine for large-scale vector data. Vald enables the similarity search for Yahoo Japan scale data. We use Vald in the recommendation platform, image search, and video analysis and text search by the vector.
Although we already have the similarity search technology called NGT/ngtd. It only supports running on a single server, and there was a problem that memory and CPU did not scale and was not suitable for distributed cloud environments.
So, for now, I’m working on developing a distributed dense vector search engine called Vald (Vektor+Wald in german)
SWAT is a task force team gathering the most skillful internal engineers. SWAT participates in various projects inside the company to resolve different internal technical problems.
The details of the key projects are mentioned below.
In-house Centralized Customer Services Platform
Architectural Adviser 2019/6~2019/8
I supported the new In-house centralized customer service platform team as an architectural adviser, and we already have over 200 CS tools in our company. It was not possible for us to improve CS quality. We cannot analyze CS data across each CS platform. Thus we needed a new CS platform that integrates the full functionality of the current CS system. We were in charge of taking care of CS data security and analytical data structure and data flow.
I have many experiences with the cloud environment and security and data analysis, and I provided numerous advice and close support for the platform.
Service Authorization Mesh Sidecar
Chief Architect & Founder (Go/Kubernetes/CloudFoundry/OpenStack) 2018/11~2019/7
In Yahoo! JAPAN, we use Athenz as an SSoT RBAC source for service authorization and authentication.
It was difficult for internal engineers to implement authorization logic using Athenz. In most cases, it took three days to implement, and it was left to the engineers to decide whether to implement Athenz.
So, I thought of creating a transparent authorization proxy that operates like ServiceMesh in the cloud environment.
I implemented it in a month, and I published it to our internal engineers.
As a result, the in-house engineer can deploy the authorization function in 30 minutes, and the mesh sidecar manages the security certificate.
It’s an open-source project written in Go.
Kubernetes as a Service Authorization Platform Garm
Engineer & Chief Architect (Go/Kubernetes) 2018/4~2018/11
I have designed and developed the Kubernetes authorization platform, which is called Garm. Base on the concept of SSoT, Garm can be integrated with Athenz and Kubernetes webhook authorization. This enables a lot of Kubernetes clusters authentication management, and Yahoo! JAPAN currently manages over 450 Kubernetes clusters by using Garm. Garm is now running on all Kubernetes clusters in Yahoo! JAPAN.
It’s an open-source project written in Go.
Approximate Neighborhood Graph Tree Vector Search Engine NGTD/Gongt
Engineer & Quality Engineer (Go/C++) 2017/10~2018/4
I developed a general-purpose dense vector search engine called “NGTD”. Users can register, search, and delete vectors in NGTD via RESTful API and gRPC at the same time, and The user can also quickly search for KNN vectors.
It can be used in a wide range of fields such as similar image retrieval and advertisement recommendation in Yahoo! JAPAN. NGTD is an open-source project written in Go.
Yahoo! JAPAN Online EBook Store
Frontend Engineer (Vue.js/Nuxt.js) 2017/12~2018/2
As a part of a large-scale project regarding E-Book Japan Corporation (a subsidiary of Yahoo! JAPAN). I developed the web frontend of the ebook store, writing in vue.js, based on atomic design.
In-house A/B Testing Platform
Engineer & Chief Architect (Go/C++/CloudFoundry) 2017/10~2018/2
I designed and developed a platform for A/B testing, which is still used internally by Yahoo! JAPAN today. The server architecture that I designed can withstand a minimum of 80000 req/sec in a microservice architecture.
The software is written in Go and running on CloudFoundry. The internal algorithm provided a high-speed and flexible A/B testing platform for users using murmur3.
Yahoo! JAPAN Premium Member Advertising Platform
Engineer & Chief Architect (Go/CloudFoundry) 2017/5~2017/7
I have developed and designed an advertising platform for Yahoo! JAPAN’s premium members.
In the advertisement distribution, I developed the platform with low-latency and high-speed delivery algorithms. Also, we use a lock-free Concurrent Map for speed. I created the entire architecture and microcomponents from scratch.
Yahoo! JAPAN Auction
Android Engineer (Kotlin/Java) 2017/3~2017/9
I joined Yahoo! JAPAN’s auction team and worked on a large-scale renovation of the Android application. I was responsible for rewriting Java to Kotlin. I made changes to the application, and daily active users are greatly increased.
Yahoo! JAPAN User Identifier Platform
Backend Engineer (C++) 2017/2~2017/3
In Yahoo! JAPAN’s user ID platform, I was in charge of adding the function of a unique cookie algorithm that manages user IDs. The library was written in C ++ and was an old source, so I added a feature with refactoring.
Yahoo! JAPAN Mail
Backend Engineer (Java/Hbase/Zookeeper) 2016/4~2017/1
Yahoo! Mail’s backend engineering was my first project after I joined SWAT.
There, I was developing/localizing the storage layer of the Yahoo! JAPAN Mail platform. On that team, I also worked as a scrum master.
The Go language technical support team develops internal Go language libraries, creates model cases, maintains CI / CD for Go, and so on.
I regularly lead study sessions to teach the other engineers about the Go language. In these workshops, all attendees from beginners to intermediates can focus on learning Go or getting better at it. At the largest study meeting, more than 200 in-house engineers attended, including myself.
In addition to SWAT, I also worked in the HR Strategy department.
I was an advisor to the Human Resources Department. As an incumbent engineer, I worked on analyzing the hiring demand of Yahoo Japan engineers by job type from two perspectives, technology and field perspective.
I was in charge of the deployment design and network construction design of the multi-service access node nationwide on the access node team. I developed the deployment prediction software creating my feedback algorithm. This algorithm can predict the number of servers so that the equipment of the ring network and construction can be adjusted accordingly.
As an intern, I gained experience in the development of EC site backend systems, in-house Q&A systems, and PoS systems. Furthermore, I also got the opportunity to be the internship team leader for the Q&A system and learned to lead a team.
glg is faster than Go standard logger and provides leveled logging and colorized output option.
Glg is used by many of Yahoo Japan’s Go language platforms.
gache is an ultra-fast lock-free cache library using a concurrent map & xxhash algorithm.
Gache is used in many of Yahoo Japan’s Go language platforms.
Vald is a highly scalable distributed high-speed approximate nearest neighbor dense vector search engine.
Yahoo! JAPAN uses vald for their similar image search engine.
Garm is an API for a Kubernetes authorization webhook that integrates with Athenz for access checks. It allows flexible resource mapping from Kubernetes resources to Athenz ones.
Yahoo Japan uses Garm and Kubernetes AccessReview Webhook to integrate and manage hundreds of large Kubernetes clusters based on the concept of SSoT.
Authorization-Proxy is Kubernetes’s sidecar container ant provides a common authentication and authorization interface for the user. which enables authentication and authorization checks for specific URL resources.
It fetches the policies from yahoo’s Athenz. Then it provides a reverse proxy interface for the user to authenticate the role token written on the request header, in order to allow or reject the user’s specific URL request.
Awards such as Hackathon or Contest are below.