CS5182 Course Project (2019/2020 Semester B) 1 Objectives The objectives of this project are for students to have some hands-on experiences for the point cloud applications. Specifically, students are required to be familiar with recent deep learning-based approaches for 3D point cloud processing. 2 Requirements of the Assignment Students should choose one of the following listed tasks to explore, including the implementation of the algorithm, and performance analysis. In order to have a fast training and evaluation process, it is highly encouraged to use Python and proceed the project on machine equipped with GPUs. List of point cloud tasks: • Point cloud shape classification • Point cloud part segmentation / scene segmentation • Point cloud upsampling • Point cloud downsampling • Point cloud registration • Point cloud LiDAR detection • Point cloud compression • Point cloud restoration, e.g., denoising and completion Other tasks are also encouraged, once they are based on deep learning. Some references are also provided in the end of this document. Students can refer to the open source code of those methods in GitHub. This assignment can be carried out as individual or group projects. The maximum number of members in each group is 3. However, the group-based assignment is expected with more work and creativities, and the responsibility of each group member should be clearly indicated in the report. There are two levels of requirements for the program, basic and advanced, to cater for students of different backgrounds and interests. The basic require- ments are designed for all students to practice deep learning methods towards point clouds, which accounts for 65 percent of the final score. The advanced requirements are for students who would like to go further to propose a novel algorithm, which accounts for 25 percent of the final score. The presentation will take 10 percent. 1 2.1 Basic Requirements (65 percent) 1. Introduction of the project Student should introduce the selected task, including a clear statement of input and output, the dataset you use, the source paper of you algorithm, the objective, the motivation, and the challenging issues. 2. Deploy deep learning environment Student should install the deep learning platform, such as TensorFlow or PyTorch on his/her own computer or remote server. One can refer to Anaconda for environment management. 3. Run Demo code You will refer to some papers for the algorithm you choose for the specific task. Most of them provide the demo code on Github. It can be a training code, or a evaluation code with the trained model provided. Students should successfully run such a demo code and describe the function of each step. 4. Train model Students are required to train a model by themselves. The corresponding dataset should be downloaded. The model may takes a dozen hours to train on GPU. And it will take longer time to train on CPU, which is not recommended. 5. Compare your experimental results with those provided by the corresponding paper Students should examine the performance of the trained model by yourself and pretrained model (if provided). The quantitative error and accuracy should be investigated. Students are also required to show the visualized results. One can refer to MeshLab to view 3D point clouds. 6. Analyze the performance on other dataset Students should pick another dataset to verify the performance and gen- erality of this method. 7. Analyze the drawbacks of the method Students should carefully examine the failure cases of the algorithm, and analyze the reason. Some ideas should be proposed to solve the problem. 8. Implement and compare with other state-of-the-art methods For the same task, students should pick another deep learning based al- gorithm, implement it and compare the results. 2.2 Advanced Requirements (25 percent) You are expected to extend this algorithm for new tasks, or try to improve it by your proposed method. You may need to modify the network structure, the loss function or dataset. Students should state clearly why such modifications are needed and show final results after such changes. 2 2.3 Presentation (10 percent) The project should be presented during the lecture/tutorial time. Each project has 15 minutes (10 presentation + 5 Q&A). The presentation will start from week 12. 3 Submission Details Each group needs to submit the following items via Canvas, and the submission link in Canvas will be open later. The submission deadline is 23 April 2020. Program: 1. A source subdirectory containing all the source files, do not include the platform source code and training dataset. 2. Txt files which contain the results for training and evaluation periods. 3. 3D point cloud data .xyz or .ply which are evaluated in your report. Report: 1. A cover that indicates your name(s) and student ID(s). 2. Describe the results listed in requirements in last section. 4 Marking This course project contributes 25% of the final course mark. 5 References Some useful links: • Search for papers: Google Scholar https://scholar.google.com.hk/ • Advanced computer vision papers: Arxiv https://arxiv.org/list/cs. CV/recent • Open source code: Github http://github.com/ • Environment management: Anaconda https://www.anaconda.com/ • TensorFlow API https://www.tensorflow.org/api_docs/python/tf/ • PyTorch API https://pytorch.org/docs/stable/index.html • 3D point cloud viewer: MeshLab http://www.meshlab.net/ Some reference papers: 3 • Deep Learning for 3D Point Clouds: A Survey • PointNet: Deep Learning on Point Sets for 3D Classification and Segmen- tation • More reference papers: https://github.com/Yochengliu/awesome-point-cloud-analysis 4
