Vision in NXT

-


The Mindsensors NXTCam-v2 vision subsystem allows users to define their object colors and track objects. Track up to 8 objects simultaneously. The object shapes could be boxes or lines. Reporting coordinates for all the objects it finds to the NXT. This is an excellent device for innovative educational courses, covering robotic vision. NXTCam can also be used as a color sensor, Roboball sensor, IR-Seeker, etc. This system ¬¬has real-time image processing capability.

Features:

• Connects to NXT on a sensor port
• Connects to computer using USB interface (USB Cable sold separately, RB-Spa-47)
• Tracks line or up to 8 objects (with 8 different user defined colors) at 30fps
• Adjustable lens focus
• Lens with built in infrared blocking filter
• Provides real-time tracking statistics to NXT
• Power consumption: 42 mA (max) at 4.7V
• PC not needed for autonomous operation on NXT
• Supported environments: NXT-G, RobotC, LeJOS, NXC
• Enclosed Design
• I2C interface with Lego Mindstorms NXT connector
• USB "Mini B" interface for optional PC operation
• Perfect for education and experimenting with robotic vision

Note: Using NXTCam requires fair amount of programming expertise and analytical skills.

Time: 4.5 hrs.

Path Following Algorithms

-


Currently I am working on developing my Leader-Follower algorithm and researching different approaches to achieve such goal. In this blog entry I want to discuss some of those approaches and the possibility to incorporate them with my algorithm.

An interesting strategy is called the “carrot following approach” where the leader transmits sub-goals to the follower and the follower uses at least two of them to locate the leader. Updates are transmitted periodically and simple mathematics and attitude estimation are used to follow the leader.

A different approach is to use controllers that produce command curvatures such as Pure Pursuit and proportional integral derivative (PID) controllers. Pure Pursuit was used by many different robots to maintain better navigations. Using Pure Pursuit with the carrot following approach has a great potential to help my algorithm. In such strategy the y position of the carrot and a specific look-ahead distance are used to locate the leader and calculate command curvature. Pure Pursuit is based on simple math but stability is a controversial. PID is more stable and uses complex mathematics but it has poor path following around curves.

Time: 5.5 hrs.

Implementing Leader-Follower Algorithm

_


While trying to implement the Leader-Follower algorithm on the NXTs using C language many different obstacles got in the way. The first challenge I faced was to get one robot to follow the other one. The robots have no since on direction and have limited memory to keep track of where the leading robot is. One of the first things that I thought about was to add a pink ball to the leading robot and have the follower have a light sensor which can locate the pink ball and follow it. Such idea proved to be inconvenient based on the fact that the light sensor get less effective as the distance increases and a the chances of losing the leader robot get higher, also the ball can fall off the robot. Adding an LED light to the robot makes it easier to find instead of having a ball. But this means that the leading robot has to be moving slower due to the fact that the follower robot has to make a 360 degree turns every time it needs to find the leader.

Another approach is to use a HiTechnic Compass Sensor which measures Earth’s magnetic field in order to provide accurate navigation. Also, it can be used to calculate the current position. This can be used by the robots to send their positions through Bluetooth or IR to help achieve the Leader- Follower algorithm. A Codatex RFID Sensor was under consideration, it’s a sensor that can read up to five-byte transponder numbers into the NXT Brick. It can be used to send directions to the follower robot but its range is very limited 30mm which defeats the goal of using it.

Another interesting idea was to use a Vision Subsystem v2 for NXT (NXTCam-v2) - This vision subsystem connects to NXT on a standard sensor port, and tracks up to 8 colored objects. It reports coordinates of each object to NXT. Vision is very complicated in the robotics world, but I believe such a sensor would help greatly in achieving our goal using the Leader-Follower algorithm. One of the disadvantages that it costs about $150.00, which I don’t know if it fits with our budget or not.


Time: 8.5 hrs.

Leader-Follower Algorithm

After doing research about the Leader-Follower algorithm I feel like I have a general understanding of the subject. Leader-Follower Algorithm is based on the fact that there is only on robot leader in the swarm at any point of time. Each robot has a preceding and a following robot. The robot has two states stooped or moving. There is one leader and the rest of the robots are followers. The leader robot is in a moving state. Once the leader robot gets in the stooped state, it is not the leader anymore; another moving robot becomes the leader. The area where the robots are moving in is to be represented as a grid of cells. Each cell represented by (x,y). A particular robot position is represented by the cell it is in at that particular time. The robots can distinguish between the cell that have been already been visited by its leading robot.

Time 6 hrs.

Algorithms and Sensors for Small Robot Path Following

-


Hogg, R.W. Rankin, A.L. Roumeliotis, S.I. McHenry, M.C. Helmick, D.M. and Bergh, C.F. Matthies, L. . "Algorithms and Sensors for Small Robot Path Following." Robotics and Automation, 2002. Proceedings. ICRA '02. IEEE International Conference on 2002. pages 3850- 3857 vol.4 Web.23 Jun 2009. .

Abstract:
“Tracked mobile robots in the 20 kg size class are under development for applications in urban reconnaissance. For efficient deployment, it is desirable for teams of robots to be able to automatically execute path following behaviors, with one or more followers tracking the path taken by a leader. The key challenges to enabling such a capability are (1) to develop sensor packages for such small robots that can accurately determine the path of the leader and (2) to develop path following algorithms for the subsequent robots. To date, we have integrated gyros, accelerometers, compass/inclinometers, odometry, and differential GPS into an effective sensing package. This article describes the sensor package, sensor processing algorithm, and path tracking algorithm we have developed for the leader/follower problem in small robots and shows the results of performance characterization of the system. We also document pragmatic lessons learned about design, construction, and electromagnetic interference issues particular to the performance of state sensors on small robots”. (Hogg and Rankin )

This article states the importance if team robotics and described it as an effective way to achieve goals that cannot be done individually. One of the main problems that occur when working with swarm robotics is dispersion. Many different strategies were introduced to achieve good dispersion but the main concentration was on the Leader-Follower strategy.

While reading this article I got more familiar with some navigation sensors such as GPS receivers and internal navigation sensors such as the internal measurement units (IMUs). Also, position estimation techniques were discussed. First, noise model for the Systron Donner Quartz gyro. Second, TCM2-50 Compass/Inclinometers Characterization. Third, Kalman filter based attitude estimation with both the Dynamic Model Replacement and the Attitude kinematics and error state equations aspects.


The article dives deeply into system architecture and Leader-Follower control. It starts by explaining the main architecture of the project, and what sensors are used. It continues on with an interesting path following control algorithm. Pure Pursuit Controller, a controller that produces command curvatures was experimented with. Also, a proportional integral derivative (PID) controller was used too. The research combines both Pure Pursuit and PID Control together and discusses the results. Outdoor and indoor testing was done, results that include problems that occurred and possible solutions.


Time 8 hrs.

Algorithms for Rapidly Dispersing Robot Swarms in Unknown Environments

-----------------------------------------------------------------------------------------------
This is a summary of the Article “Algorithms for Rapidly Dispersing Robot
Swarms in Unknown Environments”
Tien-Ruey Hsiang, Esther M. Arkin, Michael A. Bender, S´andor P. Fekete, and Joseph S. B. Mitchell

-----------------------------------------------------------------------------------------------
This paper discusses two major issues in swarm robotics, First dispersion algorithms where the robot swarm fills an environment as fast as possible. Dispersion algorithms are mainly based on two different strategies, greedy strategies and artificial physics strategies. The main concentration in the article is about Follow the Leader strategies; it discusses the work done by the writes and the results. Also it covers many different Leader-Follower algorithms.
The research was done in a simulation environment; the robots will come out of a door or more than one. Dispersion strategies for a single door such as Dept-First Leader-Follower and Breadth-First Leader-Follower were discussed in depth and compared. Also, Dispersion strategies with multiple door such Laminar Flow Lead-Follower and many different others were discussed in this paper.
This paper was very helpful in understanding the general concept of the Leader-Follower algorithms. After reading this paper I feel like I can create my own algorithm to fit the purpose of my research.

Time : 6.5 hrs

Robotics Presentation

Dr. Pearce asked me to do a presentation on Robotics for the Upward Bound student. The presentation discussed a brief history of robotics and the many different robots that were created throughout the history. The use of robotics in many different fields such as industry, astronomy, catastrophes and many others was covered. The student were to work with the RCX robots in the future, so different demos and programs were shown to the students with the RCXs. Also, NXT –a newer model of the RCX- programs were displayed. Students were shown how to use a joystick to control the NXT, by hooking up the joystick to a computer and the NXT to the computer using Bluetooth. Different short videos of robots such as Big Dog, Roomba, NXT, RCX and others also were shown to the students.

The presentation took place at Bruce Trades 226 around 11:00 AM. The students were very interested, and the presentations went well. Students asked questions and were involved in presentation effectively. I talked shortly about Swarm Robotics and my involvement in the summer research at Berea College.

Time: 14 hrs.

Follow the Leader Algorithm

After doing research about swarm robotics Dr. Pearce asked me to start off with follow the leader algorithm. At the begging I had no idea what to do or what can I do to create such algorithm, so I had to do some research in order to have a general understanding of the subject. Research was very difficult taken the fact that I was not to research about what others did because the main reason is for me to create my own algorithm without any external effect, which made understanding the subject that much harder.

I met with Dr Pearce and ideas such as having the robots follow the leading robot but adding a distinguished mark on it such as a pink ball or something like that. Such algorithms are not as effective or the type that we are looking for. As I was doing research I found an article called “A Review of Studies in Swarm Robotics”, this article looks very inserting and promising to help me have a better understanding or such Algorithms.

Time: 8 hrs.

RobotC Frequently Used Commands

As I was searching online trying to learn how to use RobotC I found a very helpful website that describes many commands for both the NXT and RXC. I went through it deleted the parts about the RXC, and went through every single step and downloaded the programs on the NXT. This was very helpful and very time consuming but you end up with a great deal of knowledge once you are done with all of it. You could also find this on the RobotC website at http://www.robotc.net/content/lego_quick/media/code_listing/nxt.html .

---

Motors
Basic motor control commands, plus some fine-tuning commands.

motor[output] = power;

NXT

Turn the referenced NXT motor output either on or off and also sets the power level of the motor outputs.

The NXT has 3 motor outputs: motorA, motorB, motorC

The NXT can handle power levels from -100 (full reverse) to 100 (full forward). A power level of 0 will cause the motors to stop.

motor[motorA] = 100; //Output A - full speed forward

motor[motorB] = 100; //Output B - full speed forward

---

bMotorFlippedMode[output] = 1 or 0;

NXT

This feature is used to reverse the direction the referenced motor travels. Once this setting is changed, the reference motor will be reversed for the entire program (unless manually changed a later point in the program)

This is useful for when working with motors that mounted in different directions and the programmer wants to keep the power setting the same for both motors.

There are two settings: 0 - normal, 1 - reversed

Before:

motor[motorA] = -100; //Output A - full speed reverse

motor[motorB] = 100; //Output B - full speed forward

After:

bMotorFlippedMode[motorA] = 1; //Motor Direction Flipped

motor[motorA] = 100; //Output A - full speed reverse

motor[motorB] = 100; //Output B - full speed forward

---

bFloatDuringInactiveMotorPWM = true or false;

NXT

This switch is used to choose wether the motors on the NXT will float or brake when there is no power applied. By default, the motors will float.

The reason to switch this is if you're trying to make precise turns with the NXT and the motors are drifting causing the change in direction to be off.

There are two settings for this switch:
false - motors will brake when inactive
true - motors will float when inactive

bFloatDuringInactiveMotorPWM = false; //motors will brake when power is set to "0"

---

Timers
The NXT allows you to use Wait commands to place delays in your program. It also supports Timers, which work like stopwatches - they count time since the last reset, and you reset them again when you want to start or restart.

wait1Msec(wait_time);

NXT

This function will cause a program to wait a specified number of milliseconds before executing the next instruction in a program.

Wait_time is an integer (1 = 1/1000th of a second). Maximum wait time is 32.768 seconds when using this function.

motor[motorA] = 100; //Output A - full speed forward

wait1Msec(2000); //Wait 2 seconds

motor[motorA] = 0; //Output A - off

---

wait10Msec(wait_time);

NXT

This function will cause a program to wait a specified number of hundredths of a second before executing the next instruction in a program.

Wait_time is an integer (1 = 1/100th of a second). Maximum wait time is 327.68 seconds when using this function.

motor[motorA] = 100; //Output A - full speed forward

wait10Msec(200); //Wait 2 seconds

motor[motorA] = 0; //Output A - off

---

time1[timer] = time;

NXT

This timer function returns the current value of the referenced timer as an integer in a specific resolution. The resolution for "time10" is in milliseconds (1 = 1/1000th of a second).

The maximum amount of time to be recorded is 32.768 seconds (~1/2 minute)

The NXT has 4 internal timers: T1, T2, T3, T4

int x; //Init variable x

x = time1[T1]; //assign variable x value of Timer #1 (1/1000th seconds)

---

time10[timer] = time;

NXT

This timer function returns the current value of the referenced timer as an integer in a specific resolution. The resolution for "time10" is in hundredths of a second (1 = 1/100th of a second).

The maximum amount of time to be recorded is 327.68 seconds (~5 minutes)

The NXT has 4 internal timers: T1, T2, T3, T4

int x; //Init variable x

x = time10[T1]; //assign variable x value of Timer #1 (1/100th seconds)

---

time100[timer] = time;

NXT

This timer function returns the current value of the referenced timer as an integer in a specific resolution. The resolution for "time100" is in tenths of a second (1 = 1/10th of a second).

The maximum amount of time to be recorded is 3276.8 seconds (~54 minutes)

The NXT has 4 internal timers: T1, T2, T3, T4

int x; //Init variable x

x = time100[T1]; //assign variable x value of Timer #1 (1/10th seconds)

---

ClearTimer(timer);

NXT

Resets the referenced timer back to zero seconds.

The NXT has 4 internal timers: T1, T2, T3, T4

ClearTimer(T1); //Clear Timer #1

---

Sensors
Sensor commands for configuration and usage are listed below. Most sensor setup should be done through the Robot > Motors and Sensors Setup menu for best results.

SetSensorType(sensor_input,sensor_type);

NXT

Sensor Type	Description	Range of Values
sensorTouch	Digital	0 to 1
sensorLightActive	Analog, Percentage ( Light Sensor with LED)	0 to 100
sensorLightInactive	Analog, Percentage (Light Sensor w/out LED)	0 to 100
sensorSoundDB	Analog, Percentage	0 to 100
sensorSONAR	Distance, CM	0 to 255

This function is used to manually set the mode of a specific input port to a specific type of sensor. We recommend, however, that you use the "Motor and Sensors Setup" wizard in RobotC.

The NXT has 4 sensor inputs: S1, S2, S3, S4

The NXT supports 8 different types of sensors:

SetSensorType(S1, sensorTouch); //Input 1 now is set to be a Touch Sensor

SensorValue(sensor_input)

NXT

SensorValue is used to read the value of the referenced sensor port. Values will correspond to the type of sensor set for that port (see set_sensor_type page).

The SensorValue function can be accessed like a variable, as it returns an integer value.

The NXT has 3 sensor inputs: S1, S2, S3, S4

SetSensorType(S1, sensorTouch); //Input 1 now is set to be a Touch Sensor

if(SensorValue(S1) == 1)

{

motor[motorA] = 100; //Output A - full speed forward

}

---

nMotorEncoder[motor]

NXT

This function is used to access the internal encoder from the NXT's motors.

A value is return with the number of degrees the motor has traveled (1 = 1 degree), with the value as an integer.

while(nMotorEncoder[motorA] <>

{

motor[motorA] = 100; //Output A - full speed forward

motor[motorB] = 100; //Output B - full speed forward

}

You can also assign the value of nMotorEncoder to 0 to reset the encoder.

To reset the encoder on MotorA, use the following code:

nMotorEncoder[MotorA] = 0;

---

ClearSensorValue(sensor_input)

NXT

This function is used to reset the value of the referenced sensor port back to zero. This is only neccessary with specific sensor types that retain their values (e.g. Encoder Sensor).

The NXT has 4 sensor inputs: S1, S2, S3, S4

SetSensorType(S1, sensorRotation); //Input #1 now is set to be a Rotation Sensor

ClearSensorValue(S1); //Reset Input #1 back to a value of 0

---

Sounds
The NXT can generate tones, or play back stored waveform sound data.

PlayTone(frequency, duration)

NXT

Plays a sound from the NXT internal speaker at a specific frequency (1 = 1 hertz) for a specific length (1 = 1/100th of a second).

PlayTone(220, 500); //Plays a 220hz tone for 1/2 second

---

PlaySound(sound_name);

NXT

Sound Names

soundBlip

soundBeepBeep

soundDownwardTones

soundUpwardTones

soundLowBuzz

soundFastUpwardTones

soundShortBlip

soundException

Plays a sound effect from the NXT internal library. Requires a sound name to be passed to play the sound.

PlaySound(soundUpwardTones); //Plays the sound "Upward Tones"

PlaySoundFile(file_name);

NXT

This function is used to play a sound file that is on the NXT. NXT sounds files have the extension .rso.

PlaySoundFile(Whoops.rso); //Plays the sound file "Woops.rso"

---

LCD Display
Commands for the NXT's LCD Display.

nxtDisplayTextLine(line_number, text, var1, var2, var3);

NXT

Displays a text line on the LCD screen of the NXT. Up to three variables are passable to the function.

line_number - The line the text line will appear on the NXT. There are 8 lines of text avaiable to the users, and valid arguments for this parameter are 0-7, which correspond to lines 1-8 on thte NXT's screen.

text - The text parameter is what shows up on the screen. This will be a string enclosed in quotes up to 16 characters. You may also display variables with this parameter by using the "%d" to display a variable definined in later parameters. You may use up to 3 "%d" characters to display 3 seperate variables. Remeber that you can only display 16 total characters, so the value of the variables will take up some of those 16 characters.

var1, var2, var3 = These parameters define which variables will be displayed to the screen.

int x = 1; //declare first variable

int y = 2; //declare second variable

int z = 3; //declare third variable

nxtDisplayTextLine(0,"Test %d %d %d",x,y,z);

//layout to display the text "Test 1 2 3" on the first line of the NXT screen

---

eraseDisplay();

NXT

Clears the NXT's LCD screen of all text and GUI images.

eraseDisplay();

---

Miscellaneous
Miscellaneous useful commands that are not part of the standard C language.

random(value);

NXT

Returns a random number (integer) between 0 and the 'value' of the function.

int x; //declares x as an integer

x = random(100); //Returns a random value between 0 and 100

---

Time: 14hrs.

How to use RobotC!!!

RobotC is a very simple program to use, it uses C-based language, very easy to get used to, and help is available inside the program and on the RobotC website.

To start things off you need to download the program from RobotC.net and then download the appropriate firmware for the NXT. Once everything is downloaded and ready to go, hook up the USB cord to the NXT and then to your computer. Turn on the NXT and open up RobotC. Go to Robot tab and choose “Download Firmware”, choose the appropriate firmware and hit open, the firmware will automatically download onto the NXT. There are many test files written in C language just open them and load them to the NXT and that way you can see what the code does to the NXT, I found that way to the best. There is descriptive and detailed documentations about building the NXT and the movement of the NXT. There is an interesting Lego Curriculum on the RobotC website.

Time: 5.5 hrs.