Profile

I am now an Algorithm Engineer of Cyberverse Localization Researsh Group at Cyberverse@Huawei (河图@华为云 ). My research work focuses on machine learning and visual localization, such as image retrieval, low-light image enhancement, image matching and so on. I received my B.S. degree from UM-SJTU Joint Institute, ShanghaiJiaoTong University, 2019; I received my M.E. degree under the supervision of Professor Yuji Yamakawa from the University of Tokyo, 2021, with a focus on flexible object manipulation, human-robot cooperation, object detection , 6D pose estimation and high-speed vision.

Education

  • The University of Tokyo
    M.S. in Mechanical Engineering | Expect: Sept. 2019 - August 2021
    GPA: 4.0/4.0
    Courses Highlights: Robot Manipulation (A), Network Optimizations (A), Active Vibration Control (A), Approximation and Online Algorithms with Applications (A), Multithreaded Parallel Computing (A), Visual Media Engineering(A), Data Science and Machine Learning(A), Machine Learning for Multimedia Processing (A).
    (The highest grade in UT is A rather than A+)
  • Shanghai Jiao Tong University (SJTU)
    The University of Michigan and Shanghai Jiao Tong University Joint Institute (UM-SJTU JI)
    B.S.E. in Mechanical Engineering | Sept. 2015 – Aug. 2019
    Major GPA: 3.93/4.0 | Major Rank: 1/58 | Overall GPA: 3.77/4.0 | Overall Rank: 2/58
    Course Highlights: Introduction to Dynamics and Vibrations (A+), Intro to Circuits (A), Linear Algebra (A+), Modeling, Analysis and Control of Dynamic Systems (A), Automatic Control (A+), Introduction to Robotics (A), Design and Manufacturing Series (A-, A-, A,A+)
  • Ritsumeikan University
    Winter Program: Janpanse Language (A) | Jan.2019 - Feb. 2019
  • Shijiazhuang No.2 High School
    High School Education | Sept. 2012 - June 2015
    Score: 700 | Rank: 59/203094 | College Entrance Examination of Science Subjects in Hebei Province.

Selected Projects

CyberLoc: Towards Accurarte Long-Term Visual Localization

Our approach wins the localization challenge of ECCV 2022 workshop on Map-based Localization for Autonomous Driving (MLAD-ECCV2022). Especially, our approach achieves 98.1% on the highest accuracy level, well over 50% ahead of the second place team.

Presented a robust and accurate mapping and localization pipeline, applied a pose graph optimization method combining absolute pose from global visual localization and relative pose from slam, designed consensus set maximization to utilize multi-session map and select the best pose track.

[video] [paper]

Dressi: A Hardware-Agnostic Differentiable Renderer with Reactive Shader Packing and Soft Rasterization

Presented a practical hardware-agnostic differentiable renderer called Dressi. Experiments and applications demonstrate that Dressi establishes hardware independence, high-quality and robust optimization with fast speed, and photorealistic rendering.

[paper] [video] [application]

HiVTac: A High-Speed Vision-Based Tactile Sensor for Precise and Real-Time Force Reconstruction with Fewer Markers

Analyzed the deformation of a thin elastic layer on a vision-based tactile sensor by establishing a simplified deformation model. A prototype HiVTac is fabricated that demonstrated superior accuracy to its vision-based tactile sensor counterparts in reconstructing an external force.

[paper]

Ring Tracking and Catching (Experiment)

Proposed a marker-based pose tracking algorithm using a high-speed vision system with marker matching and ROI fitting methods. Designed a catching point selection algorithm to make the collaborative robot accomplish high-speed catching flexibly.

[video] [paper1] [paper2]

Recognition and Catching of a Thrown Ring Using High-speed Vision in Simulation

Proposed a marker-based binocular pose estimation method in HSV color space and catching control strategy of a ring object. Due to Covid-19, the demo is implemented in webots simulation.

[poster] [video] [paper]

Mini Project: Rolling Ball Detection and Catching.

Implemented monocular positioning and robot arm control under ROS communication.

[video]

CCD Camera imaging based on Monte-Carlo Algorithm

Designed a software-based optical imaging model to analog CCD camera imaging system.

[poster] [demo]

Wheel-Leg Hybrid Robot with Novel Transformation Mechanism

Designed a vehicle robot with actively transformed three-leg wheels to pass through sand road and smooth road.

[demo]

Mini Project: Roaming Obstacle Avoidance and Chassis Path Planning

Utilized the sensor system and SLAM algorithm to plan the path and control the movement.

[demo]

Air Spinning

Designed and manufactured an exercise bike with an air purification shell to pump out air through the filtration system.

[poster] [demo]

Experience

2022

August - November

    Participated in the localization challenge of MLAD-ECCV2022.

2021

December

    🏫 Joined Cyberverse@Huawei

September

    🎓 Graduated from UTokyo

2020

December - July (2021)

    Master Thesis Research
  • Project: Marker-Based Pose Estimation for Rotating Objects using High-Speed Vision
  • Python C++ ROS OpenCV

October - June (2021)

    🏫 Internship in Huawei Tokyo research center
  • Project: 3D face reconstruction based on differentiable rendering (from single RGB image or multi view RGBD data)

May

2019

October - January (2020)

September

    🏫 Enrolled in Univ of Tokyo

August

    🎓 Graduated from SJTU

May - August

April - August

2018

Septemper - December

May - September

January - May

    Work in Yo-i Tech Corporation as an intern.
  • Participate in “BFG intelligent balance management system” project

2017

December - January (2018)

September - August (2019)

    Serve as student advisor in Academic Advising Center. (@ UM-SJTU JI)

March - May

2016

September- January (2017)

    Organize and conduct training of the 11th volunter teaching group (@ UM-SJTU JI)

July - July (2017)

    Serve as vice-captain of Mi Yuan Volunteer Group (@ UM-SJTU JI)

July - July (2017)

    Serve as vice-director of the career department in JI Student Union (@ UM-SJTU JI)

2015

December - January(2016)

September

    🏫 Enrolled in SJTU

Selected Honors

  • Huawei Future Star (March 2023)
  • Hero Individual Honors of Huawei CBG (Nov 2022)
  • First place of localization challenge of ECCV 2022 workshop on Map-based Localization for Autonomous Driving (MLAD-ECCV2022) (Nov 2022)
  • The Japanese Government's MEXT scholarship by recommendation of the University of Tokyo (2019-2021)
  • Shanghai Distinguished Graduates Award (June 2019)
  • Honorable Mention in Mathematical Contest in Modeling (Apr 2018)
  • Yu Liming Scholarship (Twice, top 0.5% in UM-SJTU JI) (Nov 2016, Nov 2017)
  • National Scholarship (Twice, top 1% in SJTU) (Oct 2017, Oct 2018)
  • Undergraduate Excellent Scholarship (Three Times) (Dec 2016, Dec 2017, Dec 2018)

My Life

  • Fans of JJLin from Singapore.
  • Love singing, cooking and skiing.
  • Begineer of guitar and photography.