So, in practical mobile robot motion planning, we want to have a nice trajectory, which does not only avoids collision with other agents, but is also kinematically feasible.

To achieve this, we can use optimization based algorithms. Given a feasible initial trajectory, and properly bootstraped costs, we are able to get the optimal trajectory with iterative linear quadratic regulation.

We're basically able to define cost of any shape, on any kinematic terms, including position, velocity and even acceleration. In iterative LQR, we find the local quadratic approximation of the cost functions, and run LQR to get the input of the next iteration. We can end the iteration based on some criterial, like absolute tolerance or relative tolerance -- if the optimization step can not change the trajectory much, we should be very close to the minima. Or the local minima :/

As shown in the gif to the left, with the zero control initialization, we found a trajectory which stayed away from the objects. Things could be trickier if we have different type of costs, or if we have worse initialization (which may drive the car off the road in the worst cases).

I recently bought an RTX 3090 graphics card. It has 24G memory, which should be able to handle STOA CNNs in 2016 or 2017. So why don't us feed a lot of images to this beast and let it learn something...

A good thing to try is super resolution for videos. Some videos, especially old ones, are often of bad resolution, and if we can recover them with ML models, it would be great.

The network structure was based on this webpage, which has ~300k params. Let's try on a single image first...

Turned out that the trained model was able to recover the image slightly better than cubic interpolation...

Well, we should also try if this works with video. Feeding 5 consecutive frames into the neural network... (See github for network details). And it did not work, the error was even larger than recovering from a single image, and that's bad. For ML systems, it happends.

Indie wechat mini games were once very popular among my friends, and I tried to implement a new game -- something bouncing between balls and bars, and sometimes attracted by black holes. Truned out to be a quite boring game, but I got the chance to implement the whole 2d physics engine, which was pretty cool.

This also exposed me to some very interesting technologies. Tencent has their own implementation of canvas interfaces and node servers, while there're multiple players in the wechat eco system, some of which offer pretty cool game engines. There're also companies which are making millions from wechat mini games. Thus, it was just too late for indie games.

I though about at least releasing it, but Tencent asked me for lots of documents, including certification of software copyrights, which can only be obtained by visiting some government agency in Beijing. I did not have any recent travel plans and did not want to bother someone to do this for me, thus it was never released.

And recently I want to set up a wechat subscription account to write something about cooking, building PCs and other stuffs happening in my life. I have to delete the wechat mini game, since it was binded to my wechat account. Here's the epitaph of Starlight Voyage.

It would be super cool to implement some algorithm on real hardware! Thus I bought a toolkit for a balance car from MiaoWLab and started building a balance car. The first step would definitely be testing the algorithm in simulation. I created the URDF file for the balance car, and tested (and tuned) my PID control algorithm with pybullet.

The next step is to implement the algorithm in C, and upload it to the embedded system. Yes, raw C, yes, upload through Keli, into a C51-like single chip microcomputer. Luckily, with the IMU and the pre-defined control interface for the wheels, it was not super hard to do this. And it worked!

In this project, we aimed to solve the problem of estimating depth information from single images. We tested our algorithms on both Kinect style data and correlating RGB image and Lidar data.

Many sorts of deep neural networks was covered in this project, but we found that the network structure from Eigen, et.al worked best. We finally settled down and used a similiar network structure.

And we achieved acceptable results on different datasets.

Origin Image	Ground Truth	Predicted Depth

We also have a pdf version of project report including different metrics.

I worked on this project to get some basic knowledge of reinforcement learning and have some fun solving problems automatically. All three problems in this project was solved using Q learning.

                
    Algorithm double_q_learning:
        initialize Q_1 = Q_2, memory_deque = []
        initialize explore_rate = 1, time_step = 0
        initialize state
        repeat:
            action = select from (random_action, argmax(state, Q_1)) with explore_rate
            new_state, reward, done = environment.step(action)
            memory_deque.append( (state, action, new_state, reward, done) )
            time_step += 1
            explore_rate = max(explore_rate * explore_rate_decay, explore_rate_min)
            if time_step % update_q2_steps == 0:
                train_batch = sample(memory_deque)
                q_val = bellman_qeuation(train_batch)
                Q_2 = Q_2 + back_propagation(q_val)
            if time_step % update_q1_steps == 0:
                Q_1 = Q_2
        until avg_game_reward > threshold
                
            

I used the template code from the UCB RL course homework to solve the Atari games. It is a deep Q learning algorithm, with some best practice, including double Q networks. The agent constantly achieved a average reward of over 20 on Pong-v0 after 4.3M of game steps.

The cart-pole problem should be the easiest and most common one. Nothing mysterious here. It was solved in 456 steps, which is a pretty good (and lucky) score for q learning.

I also developed an apple-picker game environment, a easy problem to apply deep Q learning with convolutional neural network, to test the code implementation. The problem was solved relatively easy, and should be a good start point for complex problems.

This project is a part of the final project of the University of Michigan EECS 599 Autonomous Driving course. The project was submitted, competed and rated as a Kaggle competition . Our group got 6th out of 35 different groups.

The dataset includes RGB images and correlated lidar depth information from simulated scenes. I guess it was from GTA5 or using similar game engine. A very frustrating thing in this project was that, only mid-sized cars, compact cars or SUVs were our target of detection, which means we need to distinct van and trucks, and do not report them in our results.

We transformed the lidar information to the same size of a layer of the RGB image using bilateral filter , and fine tuned the network from pretrained yolo-v2 from darknet.

You can also find our pdf report in which we reported the details of the project.

I saw an interesting image on the book Planning Algorithms, showing the path planning result for a car which can only turn left to drive through a complex environment. Thus I want to do the non-holonomic planning by myself.

So the first step should be implementing an RRT. And to achieve good execution time of fining closest nodes, it is neccesary to implement an underlying kd-tree.

Then we need a local planner which could give us a feasible solution for simple situations. It is not hard to derive a method to plan a path using the series of circles which lies to the left of the initial and terminal states. Just find the tangent circles, since we do not really need to guarantee to find a solution or find the optimal solution.

And it worked. Although since there's nothing to tell the car that we do not want to go reverse, the planned trajectory was full of segments of driving reverse :)

Generative Adversarial Network is really a cool stuff, enabling human to use machine learning to generate a series of images, which are merely distinguishable from some real images.

This project is just a toy project for GAN. I'm building a generative neural network and a discriminative neural network, to automatically generate a MNIST image which represents number 9.

The usage of all related pictures on this personal website are authorized by Mechmind robotics.