iseAuto Training Dataset
The iseAuto training dataset was composed as camera and LiDAR-projection images, which are recorded by iseAuto shuttle’s primary camera (FLIR Grasshopper3) and LiDAR (Velodyne VLP-32) sensors, respectively. The iseAuto training dateset was partitioned as four subsets: day-fair, day-rain, night-fair, night-rain. There are 2000 frames of data for each subset. In each subset, there are 600 frames that have manual-labeled annotations of vehicle and human classes. The rest of 1400 frames have the machine-made annotations that were created by the Waymo-to-iseAuto transfer learning CLFCN fusion model.
The dataset is free for downloading at HERE.
Camera images were stored as PNG files with the original resolution 4240x2824. LiDAR-projections were stored ad PKL files, where the perspective projection method was used to project the LiDAR points to the camera plane. The points that fall into the camera’s field-of-view were selected, corresponding 3D and camera coordinates were saved in pickle files. There is a detailed description about this part in paper
The vehicle and human annotations were made based on the camera images. Human annotators selected and masked the contours of vehicles as green (r,g,b = 0,255,0), humans as red (r,g,b = 255,0,0). There are also processed grey-scale annotation images in the dataset. The vehicle (green) pixels’ value changed to 1 and human (red) pixels’ value changed to 2, all the rest pixels’ value is 0. This is only for the metric calculation in testing procedure .
Annotation-rgb and Annotation-grey images were all saved as PNG files. For nighttime data, we provided lidar-camera projection images for better visualisation. The LiDAR points were upsampled and projected into the camera images, shown as following.
These images were used to figure out the objects for human annotators when it is hard to see the objects clearly in camera images because of the poor illumination condition. Please note that the same frame of LiDAR, camera, annotation, and lidar-camera-projection data follow the same name convention (‘sequence number’_’frame number’) except the extensions.
Inside the ‘splits’ folder, there are text files that contain the path of the data that was used in training, validation and testing. We emphasise again here that these are specifically for the experiments in our paper. The detailed description of the iseAuto training dataset split can be found in paper. The ‘norm_data’ folder contains the information that we used to normalise the Waymo and iseAuto LiDAR data. As analysed in our paper, this is a critical process for our models to get reliable output.
The bag files of all subsets are sorted by their creation sequences, which correspond to the sequence orders of the PNG and PKL files.
Camera-LiDAR calibration information
The camera-LiDAR calibration of the iseAuto shuttle was done by Autoware Camera-LiDAR Calibration Package. We have dedicated several calibrations to make sure the result is as accurate as possible. The camera-LiDAR extrinsic calibration result is as following.
Camera intrinsic matrices can be found in ROS topic ‘/front_camera/camera_info’, which are available in all bag files. The detail is as following:
