Code tutorials

Under this section I will share code tutorials for different interesting features. If you have any ideas or requests, feel free to contact me via email.

2/9/2023 - IMU

IMU sensors, which measure linear and angular accelerations along with orientation changes, play a pivotal role in generating realistic sensor data for testing and refining autonomous driving algorithms. CARLA's integration of IMU sensors empowers developers to replicate real-world scenarios, fine-tune control algorithms, and validate their systems in a virtual environment, fostering the advancement of safe and efficient autonomous vehicles. Understanding IMU sensors and their synergy with CARLA is paramount for those engaged in autonomous driving technology research and development.A new code tutorial about initializing and using IMU sensor in Carla simulator has been uploaded to the blog's GitHub repository.
As always, if you have any specific requests or ideas for cool sensors you can program or insights about this section, feel free to contact me via email.

31/8/2023 - Visualize multiple sensors

Carla's repository has an example code for the visualization of few sensors at the same time. I added a few modifications for this code in order for it to assist on data collection. Now it runs 3 sensors synchronously: RGB camera, semantic segmentation and a raw depth camera. These images are saved under 'raw' folder and can be processed for future use. In previous posts I suggested using semantic segmentation for bounding box extraction for object detection task. Also, we had a code snippet for the conversion between the raw depth image to a grayscale int24 depth image in centimeters which corrects a mistake which was made at the original carla repository.In addition, I imported keys to the code, offering the usage of different cheats. For example, one can add manual driving keys (left, right, accelerate, stop etc.) I added the option to use 'r' key for a random car location change. Also, to collect data from different parts of the city, i used a running index which counts steps, and every 50 steps it randomizes the vehicle location.Check out the code at the Carla blog github repository.If you have any specific requests or ideas for cool sensors you can program or insights about this section, feel free to contact me via email.Update (1/9/2023): I added keys to the code: toggling on and off autopilot, pivoting left and right, turning left and right (and disable the turning) and spawning driving cars inside scene.

29/8/2023 - Sensors

For a start, Enter the sensors page at Carla's read-the-docs and have read about the different sensors which can be used out-of-the-box.
Additional sensors can be made by you using the combination of data extracted from these sensors. For example, a combination of the texture from the RGB camera and the depth of each pixel from the depth camera, LIDAR combined with semantic segmentation for a more realistic LIDAR view (semantic segmentation offers us to mark down object with a lower reflectivity and therefore these will not be seen clearly at a 'real' LIDAR image.Using the proper sensor can help you perform the right task. For example - using semantic segmentation can spot an object from some class (for example - cars). Use the semantic segmentation sensor and the RGB camera at the same time and create a dataset for cars detection/segmentation. Extract a bounding box for each object from a semantic segmentation image is simple with the following codeAdditional trick is using the raw depth sensor, which offers the saving of an int24 images instead of int8. Once saving processed depth images, the data which is saved comes at a int8 format, means a very low quality. Using the raw depth format (example) offers the usage of a far more useful data.
Pay attention that the formula for converting the raw depth format to a grayscale image at the sensors page has 2 mistakes, and the right conversion is shown at the attached code.If you have any specific requests or ideas for cool sensors you can program or insights about this section, feel free to contact me via email.

29/8/2023 - Where to start

First of all, Carla has great code examples for multiple applications. I suggest starting with operating the code examples and try to play with them. Start with the following code examples:
1. Manual control
2. Visualize multiple sensors
3. generate traffic
4. dynamic weather