CHA Jérôme (チャ。ジェローム) Tutors (チューター): Mr Hirakawa, Mr Osakawa, Mr Kamaya, Mr Tezuka, Mr Kobayashi, Mr Sasaki, Mr Lebegue

Navigate for Mobile Robot with Android Devices Hachinohe National College of Technologies (八戸工業高等専門学校)

2011-2012

Acknowledgements First of all, I would like to thank all of the personals of University Lille 1 and of Hachinohe kosen because they helped me to do this internship in Japan with the best conditions. Furthermore, I thank all of my teachers, with their teachings and their abilities I was able to do my project in as good as possible. Another person is very important for me, it’s Mr Hirakawa. He always helped us during this internship and we visited a lot of Japan’s place with him. Then, if I have been completely integrated in this kosen, it is thank to my Japanese teachers Mr Rigal and Ms Shoko in France. Without them I would not be able to communicate with the Japaneses students and teachers. Finally, I would like to thank all of my tutors. They lead me perfectly. Thanks for all of foreign and Japanese students I’ve met since this three month too. They receive me with the best care, I’ve never seen before. And then, I thank all people I’ve met during this internship.

1

Abstract To begin this abstract, I’ve chosen to do this internship in Japan, it was because I love Japanese culture and I would like to see another way of life. I want to discover this beautiful country and meet a lot of japaneses students. During these three months, I’ve realized a project about an Android device and robotics. Before, a lot of projects were proposed to me. I’ve chosen this project because I want to work about Android devices in the future. The name of my project is “Navigation for Mobile Robot with Android Devices”. My tutors, Mr Kamaya and Mr Tezuka let me the opportunity to choose the main programming language. I’ve chosen Java, because Android programming is only in Java and my knowledge about this language is better than C or C++ language. That were the reasons why, I chose to do my entire project in Java. In fact, the goal of this project is to drive a robot with an android tablet. This robot would not crash on a wall. First of all, I had to send some messages to a server with my device based on android’s accelerometers. The next part, it was to move the robot with the messages. The last one was how to don’t crash on the wall.

2

Résumé Pour commencer ce résumé, je tenais tout d’abord à dire la raison pour laquelle j’ai choisis de faire mon stage au Japon. Tout d’abord, c’est parce que j’adore la culture japonaise, et j’avais envie de découvrir leur façon de vivre. D’autre part, c’est aussi parce que je voulais découvrir ce merveilleux pays et rencontrer plein d’étudiants japonais. Ensuite, durant ces trois mois, j’ai réalisé un projet portant sur la technologie Android, et la robotique. Au départ, de nombreux projets m’ont été proposés. J’ai choisis de travailler sur l’Android parce que je souhaiterai plus tard, si possible, m’orienté dans ce domaine. Ainsi donc, le nom de mon projet s’intitule « Navigation for Mobile Robot with Android Devices ». Mes tuteurs monsieur Kamaya et monsieur Tezuka, m’ont laissé le choix du langage de programmation. J’ai donc choisis le Java parce que mes connaissances en ce langage sont beaucoup plus approfondies par rapport au langage C ou bien C++. C’est donc la raison pour laquelle mon projet fut programmé entièrement en Java. Enfin, le but de mon projet était de piloter un robot avec une tablette tactile Android. Ce robot devait contourner les obstacles si ceux-ci s’opposaient à lui. Pour mener à bien ce projet, j’ai dû me baser sur les accéléromètres de la tablette et envoyer un message à un serveur, que j’ai dû développer, en fonction de l’inclinaison de celle-ci. Ensuite, j’ai dû prendre en compte les obstacles afin que mon robot puisse rouler sans forcément rentrer dans les obstacles.

3

Table of contents Acknowledgements ................................................................................................................................. 1 Abstract ................................................................................................................................................... 2 Résumé .................................................................................................................................................... 3 Table of contents ..................................................................................................................................... 4 Introduction............................................................................................................................................. 7 Hachinohe National College of Technologies .......................................................................................... 8 I/ History (歴史)................................................................................................................................... 8 1)

Aomori Prefecture (青森県) ................................................................................................... 8

2)

Hachinohe City (八戸市)......................................................................................................... 8

3)

Hachinohe Kosen (八戸工業高等専門学校) ......................................................................... 9

II/ The system of education ................................................................................................................. 9 1)

In school .................................................................................................................................. 9

2)

My Laboratory ......................................................................................................................... 9

Project (研究) ........................................................................................................................................ 10 I / Step by Step robot will move....................................................................................................... 10 1)

Value of the sensors ............................................................................................................. 10

2)

Establish a connection with a Server and a Client ............................................................... 11

3)

Display a message to the server with the sensors values ................................................... 13

4)

Moving the robot by sending message to the server .......................................................... 13

5)

No crash ................................................................................................................................ 14

II/ Next parts...................................................................................................................................... 15 1)

Display value of sonars in the pad ......................................................................................... 15

2)

The « Map drawing »............................................................................................................. 16

i) MVC pattern .......................................................................................................... 16 ii) 3)

This pattern in action .......................................................................................... 17

Android application ............................................................................................................... 17

i) Manual drive .......................................................................................................... 18 ii)

Test Connections ................................................................................................ 18

Conclusion ............................................................................................................................................. 19 Appendices ............................................................................................................................................ 20 Appendices 1: Towada Lake .............................................................................................................. 20

4

Appendices 2: Sakura’s tree .............................................................................................................. 20 Appendices 3: Shinkansen ................................................................................................................. 21 Appendices 4: The philosophy and policy of education at Hachinohe Kosen................................... 21 Appendices 5: Set the different values ............................................................................................. 21 Appendices 6: Update the TextViews ............................................................................................... 22 Appendices 7: Server/Client communication (Java programming language) ................................... 22 Appendices 8: SendMessage.java – Constructor .............................................................................. 22 Appendices 9: Axes in Java ................................................................................................................ 22 Appendices 10: MVC pattern ............................................................................................................ 23 Appendices 11: The 2 cases of axes’ positions when the robot began to move. ............................. 23 Appendices 12: Wall in front of the robot ........................................................................................ 23 Appendices 13: Robot and its sonars number .................................................................................. 24 Appendices 14: Tablet Lenovo A1-07 ................................................................................................ 24 Android application .......................................................................................................................... 25 Appendices 15: Android apps – Main menu ................................................................................. 25 Appendices 16: Android apps – Manual activity (speed normal) ................................................. 25 Appendices 17: Android apps – Manual activity (speed fast) ....................................................... 25 Appendices 18: Android apps – Test connections ........................................................................ 26 Appendices 19: Android apps – Send Message to server ............................................................. 26 Appendices 20: Android apps – AutomaticActivity ....................................................................... 27 Android Programs ............................................................................................................................. 27 Appendices 21: AutomaticActivity.java......................................................................................... 27 Appendices 22: ManualActivity.java ............................................................................................. 34 Appendices 23: RobotControlActivity.java (main class) ................................................................ 36 Appendices 24: SendMessageActivity.java ................................................................................... 37 Appendices 25: SensDisplayActivity.java ...................................................................................... 39 Appendices 26: TestConnectionActivity.java ................................................................................ 41 Appendices 27: ManualSendListener.java .................................................................................... 43 Appendices 28: CancelListener.java .............................................................................................. 44 Appendices 29: ChangeActivityListener.java................................................................................. 45 Appendices 30: SendMessage.java ............................................................................................... 46 Appendices 31: ServerMoving.java ............................................................................................... 49 Appendices 32: NoCrash.java ........................................................................................................ 53 Appendices 33: StockSonarRange.java ......................................................................................... 56

5

Appendices 34: ServerSendValues.java......................................................................................... 57 Appendices 35: R.java (Generate by eclipse) ................................................................................ 58 Android XML: .................................................................................................................................... 61 Appendices 36: AndroidManifest.xml ........................................................................................... 61 Appendices 37: string.xml ............................................................................................................. 62 Appendices 38: automatic.xml ...................................................................................................... 63 Appendices 39: displaysensor.xml ................................................................................................ 65 Appendices 40: main.xml .............................................................................................................. 67 Appendices 41: manual.xml .......................................................................................................... 68 Appendices 42: sendmessage.xml................................................................................................. 69 Appendices 43: testconnect.xml ................................................................................................... 71 Laboratory ......................................................................................................................................... 73 Appendices 44: Laboratory room. ................................................................................................. 73 Appendices 45: Robots in the laboratory ...................................................................................... 73 Others:............................................................................................................................................... 74 Appendices 46: Kabushima Shrine ................................................................................................ 74 Appendices 47: Japaneses Umineko (海猫: cats of the sea)......................................................... 74 Appendices 48: Server multiclients ............................................................................................... 75 Appendices 49: Robot and its parts .............................................................................................. 75 Appendices 50: All movements ..................................................................................................... 75 Glossary ................................................................................................................................................. 76 Bibliography........................................................................................................................................... 78

6

Introduction In order to graduate my diploma of the IUT1 Lille 1, I have to make an intern in a foreign country. I have chosen Japan because I want to know another culture and way of life. The goal of my project is to be able to move a robot with an Android device. I have decided to realize it in Java programming language because my knowledge of this language is better than C or C++. Moreover, Android’s programs can only be written in Java so it was better for me to program in only one programming language. To realize this project, I have used Aria’s libraries and software. Aria was essentially based on C++ language that is the reason why I need to use Aria.jar and ArNetworking.jar to be able to move the robot. Therefore the first library is to be able to use all Aria’s methods, and the last is to be able to connect with the robot and in MobileSim. This software can emulate a robot on a map we can also do several tests with it instead of launch in the robot directly. The Android device that I have used is a Lenovo tablet a1-07 with the version of android 2.3.4. I will do my entire program in the software Eclipse and its Android SDK2. I have chosen the following problematic. How to move a robot with Android in Java by using Aria’s libraries? First of all, I will start to speak about Hachinohe history and then I will explain my project by describing each step I have done. Finally, I will finish with the conclusion, we will speak about the problems I have encountered and the news skills and abilities I have earned.

1

In France: “Institut Universitaire de Technologie” it is a kind of French university. System Development Kit: is typically a set of software development tools that allows for the creation of applications for certain software. 2

7

Hachinohe National College of Technologies I/ History (歴史) 1) Aomori Prefecture (青森県) Aomori Prefecture is situated in the Tohoku region in the north of Japan. The number of people is around 1 363 711. This number equals about 1% of the Japan’s population. This place is very famous for apples. Indeed, it is the leader of the national agriculture production. We can see everywhere some memories or some kind of foods with the Aomori’s apples. Moreover, Japaneses call it Aomori because the kanji means “The blue Forest”. They use sometimes the color “blue” with green object for example with green apples. Japaneses says “Blue apples” (青いりんご). That’s why the truth meaning is “The green forest” because there is a lot of forest in Aomori. Furthermore, in this prefecture, there are many beautiful environmental places to visit for example, the famous Towada Lake3 and beautiful cherry blossom4. 2) Hachinohe City (八戸市) Hachinohe City is in the north of Japan with about 330 000 people. This town is located in the southeastern Aomori in the Tohoku region of Japan. From December 2002, the northern terminal of the Tohoku Shinkansen5 has been at Hachinohe station, connecting it to Tokyo Station. Its harbour is very famous but the 11th of March 2011 it was struck by the strong tsunami. It tossed many huge fishing boats ashore and heavily damaged the port area. About 100 homes were destroyed. Some countries came to help them to re-build all. Actually, this city is still a tourism place. We can find a lot of commercial places like the famous Piado or Rapia, a lot of famous restaurant and many hotels. Moreover, we can a lot of beautiful Shinto shrine like Kabushima Shrine6 that also serves as a habitat for thousand Black-tailed Gulls. Hachinohe people call them umineko7 (海猫: cat of the sea) because their screams are like a cat and they live near the sea.

3

Lake surrounding by mountains ( appendices 1) Japaneses cherry blossom are very famous(appendices 2) 5 King of speed train, like French’s TGV but speeder (appendices 3) 6 Appendices 46 7 Appendices 47 4

8

3) Hachinohe Kosen (八戸工業高等専門学校) In 1963, the first national institute of higher education for engineering was created in Aomori Prefecture. It was Hachinohe National College of Technology. Since then, more than 6000 students have graduated in this Kosen. This school was founded to “develop creative engineers through five-year education”. This college is very famous because it trains the students to be proficient technical expert. In 2002, the advanced two years course was established to obtain a bachelor’s degree. The students graduate can very easily find a good job not only in Aomori prefecture, but in Tokyo, Nagoya and Osaka regions too. In this school, the abilities of the students are teach by outstanding professors who are highly evaluated for their advanced research activities and development of new industrial products. The students promote are accredited to be on international level of engineering education by the “Japan Accreditation Board for Engineering Education”. This license guarantees the high quality of engineering education of this kosen. Hachinohe national college of technologies has a lot of relationship with international high school for example, the IUT Lille1, IUT Aix-en-Provence in France, INTEC Universiti Teknologi MARA in Malaysia and Misawa Base in USA.

II/ The system of education 1) In school In Hachinohe Kosen8, the students are educated to be more efficient in the professional world. In order to realize it, they are encouraged to study five years in this college in one of the four departments. There are mechanical engineering, electrical and computer engineering, chemical and biological engineering. All along these years, the students have to do some project and experiments in one of all the Kosen’s Laboratories. After these five years, they can continue in Advance course for two years to improve their knowledge. In the first year of this course, the students have the opportunity to do an internship in a foreign country, like in France for example. With the diploma of Advance course, more of them go to work for a company but some students prefer go to Tokyo’s university to have a better job in the future. 2) My Laboratory During these three months I studied in Mr. Kamaya’s Laboratory with some student like Mr. Tezuka, M. Kobayashi; Mr. Sasaki and others. All things I have needed, I can find it in this experimental room. Everyone is very friendly, and if I have a problem there is always someone to help me. In this laboratory9, the main theme is robotics. We can find some robots inside. Some projects are about robots like this project for example.

8 9

Go to appendices 4 Appendices 44

9

Project (研究) I / Step by Step robot will move To realize my project in the best conditions, my tutors gave me some steps to follow to do it in the best conditions. These steps helped me to do my project without a lack of knowledge. 1) Value of the sensors This step was the first that I have done. The aim is to display the value of each sensor but Android device have many king of sensor. The accelerometer sensors will calculate the acceleration with the gravity in the three axes X, Y and Z. Its unity is the m/s². Then the gyroscope sensors will the rotation around each axe. It will calculate in rad/s. Moreover, the light sensors will show us the light with the Lux unity. Next the magnetic field sensors will calculate the magnetic field with the µTesla unity. We can find pressure sensors which will calculate the pressure in KPascal. There are the temperature sensors, the proximity sensors which will show us the distance between an object and the device. Finally we have the orientation sensors to calculate the orientation of the tablet. In this project, we have just used the accelerometer sensor because this tablet doesn’t have orientation sensors. Now, the type of sensors is not enough we have to know that the tablet can be moved in three axes X, Y and Z. The particularity of Android, it is because each axe has a name. We have the azimuth axe, the roll axe and the pitch axe.

10

To display each value on the tablet, I use the object TextView of Android API10, so I have created it, one for one value. In Android program, an R class 11 was automatically generated. Inside of this, we can find all components’ ids.

In Java, to activate the sensors we have to use a SensorManager from Android API. This sensor manager will enable or disable the sensors. To implements this sensors I use the following code.

When the values change, the method onSensorChanged(SensorEvent event) will be launch. As it was written, there is one parameter. Consequently, when the values changed they are stock in an array of double. The order of number which will stock in this array is the azimuth, after the pitch and the last the roll value12. Now we had the value, we just need to display the values. To do this I just changed the text of the TextView by the corresponding value.13 2) Establish a connection with a Server and a Client There is a lot of way to do a server and a client. We can use many protocols but the two mains are TCP/IP14 and UDP15. They are very different. The protocol UDP can be used without any connection. It functions in the disconnected mode whereas the protocol TCP/IP functions in the connected mode. My servers and clients are created with the first protocol. It is very reliable and more powerful because we can manage the client as we wish. We are in connected mode, so as I want I can eject the client by closing the link between them. So, the aim of this step is to send a sentence from the client (the Android tablet), for example “hello world”, to the server (the laptop) and it will display these words. The server and the client can send some data by using socket16. It is in connected mode, so we need to use the method accept() in the server’s program to wait for a client’s connection.

10

Application Programming Interface: specification intended to be used as an interface by software components to communicate with each other. 11 See in the appendices 35. 12 See more in appendices 5 13 Appendices 6 14 Transmission Control Protocol: this protocol is in the layer “Transport” of OSI layers. 15 User Datagram Protocol: One of the core members of the internet protocol. It works in the disconnected mode. 16 It’s an endpoint of an inter-process communication flow across a computer network.

11

To continue, the server’s socket as well as the client’s socket can communicate17 by using Java’s Streams object like getInputStream() or a getOutputStream(). So if we want to send some information, we will use the input methods and the other to receive this data. Furthermore, to write the message to the server and read it we had to do like the following example. Server’s side: Client’s side: One the one hand, at the client’s side, to display a message in the server’s frame, we have to use a PrintWriter. This class implements all of the print methods found in the PrintStream class. We can use it like a common print but when we create this object we need to specify the socket. That’s why it will print the words directly to the server. On the other hand, at the server’s side, we need to create the object BufferReader to read the sending message. After create that, we need to use the method readLine() to read the message of the client’s print. Server’s side: receive the message.

Client’s side: sending the message.

With this setup the server can only accept one client. That’s mean we cannot connect two android device on the server. To delete this problem, we just need to launch the server in a Thread. The accept method of the server, when we use it, the server will lock until there is a client. So when the server is exchanging some data with another server, nobody can connect to it because the program is not finished. In this case, even if we have only the tablet as a client, it is better to launch the client in a thread to skip this problem.

In fact, if we want to receive many messages, the server needs to be always launched. So we have to make a while(true) to solve this problem18. With that, the server will continue to receive the newest client’s messages. If we don’t use that, the server will stop after receiving just one message but that is not what we want. Our server is now Multiclient.19 Furthermore, if we use an infinite loop, the server will crashed because it will use more memories. To solve this, we had to make a sleep to slow it down. Now we have the client and server, the next part is just a mix with the first and this step. 17

See appendices 7. See in appendices 31 19 See appendices 48 18

12

3) Display a message to the server with the sensors values The goal of this step is to display a letter in terms of the tablet’s inclinations. So we need to display the letter F for Forward, L for Left, R for Right and B for Back20. To solve this problem, I created a SendMessage class 21 . This object will send a message with four parameters. There are the message, the server’s IP address, the server’s port and the name of the activity which send the sentence. With this class, we can easily send a message to the server. We just need to create this object buy giving the recommended parameters. This program is in client’s side and it is launched in a thread to unlock the multi sending.

4) Moving the robot by sending message to the server To begin, we had to know how to how can we do to move this robot22. As we had saw in these last parts, we will use the last program to solve this problem. To continue, we had to know how to move the robot just by coding. To move the robot forward we need to use the function setVel(int) from ArRobot class. This function will set the velocity of the robot so more this parameter is high, more the speed will be.

Then, we need to move the robot to the left or the right. To do this instruction, the setRotVel(int) was created to this movement. This int is the rotation of the robot in degrees. A negative int means negative degrees. Here the norms are to rotate in 60 degrees if we want to go on the left, and -60 degrees for the right.

To finish, after going forward, turn on the left or right, our robot need to stop for a moment. So the answer at that problem is the method stop() from ArRobot class. It will stop the robot that means it can move again if we wish. It will not disconnect the robot from all programs.

As we know, we need to launch the server in a thread to let the multi connection and in an infinite loop. If we want to disconnect the robot by sending the message, we need to change the true in the server’s loop and use a Boolean. For example, we create a loop until the variable end is false. When we will receive the message B+ this Boolean will change at true. So we will quit this loop and disconnect to robot23.

20

Appendices 50 Appendices 8 22 Appendices 45 23 Appendices 31 21

13

5) No crash The target of the last part is to evade the obstacles. This part was probably the biggest. We need to realize a little artificial intelligence which will improve our robot’s moving capacity. We had to think like the robot. For example, is there is a wall forward and on the right, we had to move on the left. But now, if there is wall only forward, where should we go? Many problems were presents and any cases to do that. First of all, the robot can detect all things with its sonars. These sonars can recognize something about 5000 millimetres far away. The robot has 16 sonars. 8 sonars are used for moving forward and the others are to move behind. In the entire project, we will use only the eight sonars because we just need to move the robot forward. In these 8 detectors, there are two parts. Four of them are used to detect the right part, and the other the left part. Each of them has a number. It starts at 0 on the left and finish at 7 on the right24. So if we want to know the distance of the one sonar, we will use getSonarRange(int) of ArRobot class. This function will return the distant of the parameter int. It represents the number of the sonar.

Then, we can know when an object is near our robot. If a value is under the maximum distance (5000 mm) it means that an obstacle is coming soon. We have to select a minimum value, indeed, if the distance is less than our minimum value, we need to turn. For example, if the distance is less than 1000, we should to turn on the left or right it depends on the number of sensor. Furthermore, with only this condition our robot wasn’t good yet but it will evade a little. The next problem the robot met was if a wall is on the right and on the left. The distances between the two walls are under 1000. Also, our robot will move on the left or on the right or do a strange move. For example, if the value of the sonar 0 and 7 are equals to 500 but our limit are 600. So the robot will move on the right until the value of the first sonar is more than 600. Next, the value of the last sonar is under 600 too, so it will move on the left until it was more than 600. This action will continue again and over again. To avoid this, we need to improve more this little AI25. As it was written before, the maximum distance is 5000mm. So I decide to make a percentage for the left part and the right part 26 . Each part has four sonars, so the left and the right part have respectively 100 percentage. Now we will say to our robot if the percentage of the left part is more than the right part, we need to turn on the left or inversely. This system was very interesting to improve the movement. Finally, there is another case which will be very important. What is happening when the wall is in front of its. The values 3 and 4 27 will be less than the minimum limit and each part have the same percentage28.

24

See the picture in the appendices 13 Artificial Intelligence 26 Appendices 49 27 The values of the sonar 3 and 4 28 Appendices 12 25

14

So to resolve this, we need to say which part will dominate in equal case. I decided to declare the right part. Now our robot, in this case or with an impasse, our robot can move not very well but with a minimum of knowledge.

II/ Next parts 1) Display value of sonars in the pad With this program, we may move the robot without seeing him directly. Indeed, we had to display each value in a text field then we need to change them, to address that we had to find a way to set the text in the best case. Thus, we can see when an obstacle is coming, it was like a camera above its. First of all, I’d tried to launch a thread when a direction’s message was sent. This thread will send a message with the number of the sonar (for example: “0”) and the server will return the value of the previous sonar. It might be efficient but when we sent a message we used the SendMessage class, so this is this class which launched the previous thread. The question is where I can change the text to display the values. I’d tried several cases, one of these is to cast29 the parameter Activity of this sending class to the AutomaticActivity30 and use the text view method setText(String) to change the text.

Usually, it works, but we are in Android programming, so a Thread class cannot change an Activity’s TextView. As a result, I’ve tried to change set the text in the SendMessage class not in the thread. That was a little good but not really what we want. When we send for example the message “R” to turn right, with this program we will send “0”, “1”,…,”7” and receive all data from the server. It was very difficult to the client and the server to repeat each action eight times after the first message. Thence, I used encrypted messages that mean they will be sending to the server but not displayed and it will have a best performing. We don’t have to forget that we have with an Android device not with a PC so the more efficient program is obviously used. Now instead of sending several messages we will send only one. I have chosen “01234567” but we can choose “Send me values please” for example. Consequently our server will send an encrypted message to our client too. Instead of sending the values one by one as before, we'll send everything in one hit by linking values with “-“. 5000-5000-5000-5000-5000-5000-5000-5000 In this way, we can get values more easily than before. Everything we just need to do is create a function that let us to obtain each of these values and put them in an array to be able to use thereafter. The following problem is changing the text because if we decide to do it by casting the Activity in parameter, we cannot touch to the TextView too. We will have the

29

In the Java programming language, the casting/coercing operation is used to convert between types and the instanceof operator is used to check for type information at run time. 30 Appendices 21

15

error “Only the original thread that created a view hierarchy can touch its views.” So we need to do it in another way. To do it in the best condition, I’ve decided to create a function in the AutomaticActivity which change the text of the text view by giving in parameters the numberOfSonar and the value. With that, this will be the activity which will set the text. Therefore our problem is definitely solved. 2) The « Map drawing » To start, this program will draw the map in a JFrame31 according to the movement of the robot. That means at the beginning we need to create a frame with a white background and the robot in the middle. When the program start the walls and object our robot will detect will be drawing. I decided to use the MVC32 to do that. i)

MVC pattern33

The java’s Model-View-Controller pattern is a very powerful framework. As it is called, MVC was broken down into three parts. First of all, we have the Model which extends the Observer class. Then the View which implements the Observable interface and to finish we have the Controller which is primarily composed by Listeners class. Moreover, Models represent data and the rules that govern access to and updates of this data. It can notify all observers when data changed by using setChanged() and notifyObserver() methods. Views will render the contents of a model. When the previous functions were called, it activates the update() method. Then the view must update its presentation as needed. Controllers translate the user’s interactions with the view into actions that the model will perform. It could be for example a click on a button. That is the reason why, it was composed by Listener class. In fact we will separate all data in several classes, it will be easier to understand and to code because if we want to add a JFrame with our model, we just need to use addObserver() from Observable class. This function will stock into an array all observers in order to call the update method from all of them.

31

Java’s class to create a frame in the screen Model View Controllers: Java’s pattern to easily do graphical objects. 33 See appendices 10 32

16

ii)

This pattern in action

In this case, my models were the JFrame and the JPanel where we drew. My model was a class with the sonars’ values, the X and Y position of the p3dx 34 . There aren’t controllers because we don’t need user’s interactions. With this framework, it seems easy to complete this part but there is one problem that I couldn’t solve. When the robot starts, the two axes depend of the robot position 35 . We cannot determinate which position we will have. That is why when I try to draw the map I can see some strange drawing. In Java the two axes started at the top of the left hand corner and finished at the top of right hand corner for X and at the bottom for the Y axe.36 That is the reason why this part was not finished. Maybe later I will finish it, it will be my future work. 3) Android application37 At the beginning of the project, I started to create one application for one step. But at the middle of these three months I began to make only one program with a main menu38. In Android programming, an Activity is like a JPanel in Java. We can put some button, pictures, and more. With the Android API we can do it with a simple Drag and Drop but normally each activity is coding by using xml39 files. To start we need to choose our layout before to put the components. Once all needed things were put, we had to initialize them in the Java’s program before to continue. For the final application with all of my research programs I decide to make some version. I have decided to start with a version 1.0 and each time I have solve one problem I go to the 1.1, and etc…The aim to do this is because when I made 97% of tests in Android emulator, so sometimes I didn’t remember what is the last version I have put in the pad. So now I could know where I was stopped the last time. I have decided to make some programs which are not in the previous step because I wasn’t very useful but with that I will be easier to make several tests.

34

Robot’s model See Picture in appendices 11 36 See appendices 9 37 Appendices 15 to 20 is the Android application 38 See in appendices 15 39 Extensible Markup Language: it is a markup that defines a set of rules for encoding documents in a format that a human and a pc can read. 35

17

i)

Manual drive40

As I called it, this program was created to be able to drive the robot manually. One click on the button will sent one message. When the speed ToogleButton is enabling, we will activate the speed mode. This mod will activate the button “F+”. This button will send to the server the information to upgrade the speed. With this kind of activity is that I don’t need to go to the sending program and send letter by letter all I want to do. In fact, this activity was like a joystick and very useful. ii)

Test Connections41

The main theme of this activity is just to send a message “Test” to the program to see if the server is connected. If it’s works, the server will send to the client the message “OK”. When I decide to developed its, it was when I need to send an encrypted message to the server. I need to see If when send any messages, the server will send to me the encoding message with all sonars’ values. To do this, we just need to press one button, so it was easier for me.

40 41

Appendices 16 and 17 Appendices 18

18

Conclusion First of all, all along this project I have encountered many problems. The Aria’s libraries cannot be used with a Windows 64bits and my laptop is based on this kind of OS42. So I had to do something to solve it. I decided to do my entire project in Kubuntu43 64 bits I prefer do that than emulated a virtual operating system. The next problems I have met are the map drawing and the error I have written 1)44 . I have never made it before, so it takes me a long time to try to understand but finally I have done it Then, during these three months I have done a really interesting project about Android like I wanted. In the future I want to work about video games and android that is the reason why I have chosen this project. I’m very glad because all of my tutors and students I have met were always encourage me. In fact, this internship in Japan was a very good opportunity. I have improved my Japanese and I have met very nice people. Doing a project in this country is a very good experience, now I know how to explain a problem with people who speak a foreign language. With this experience rich in knowledge I can make any project without being scared. Doing this project in Japan was the best think I have never do and maybe I can come again in Japan, to work in a Japanese company.

42

Operating Systems (like Linux, Windows, Mac…) A UNIX operating system (Ubuntu). The “K” means “KDE” it is just a graphical layout but the OS is the same. 44 Click on the word 43

19

Appendices Appendices 1: Towada Lake

Appendices 2: Sakura’s tree

20

Appendices 3: Shinkansen

Appendices 4: The philosophy and policy of education at Hachinohe Kosen

Appendices 5: Set the different values

21

Appendices 6: Update the TextViews

Appendices 7: Server/Client communication (Java programming language)

Appendices 8: SendMessage.java – Constructor

Appendices 9: Axes in Java

22

Appendices 10: MVC pattern

Appendices 11: The 2 cases of axes’ positions when the robot began to move.

Appendices 12: Wall in front of the robot

23

Appendices 13: Robot and its sonars number

Appendices 14: Tablet Lenovo A1-07

24

Android application Appendices 15: Android apps – Main menu

Appendices 16: Android apps – Manual activity (speed normal)

Appendices 17: Android apps – Manual activity (speed fast)

25

Appendices 18: Android apps – Test connections

Appendices 19: Android apps – Send Message to server

26

Appendices 20: Android apps – AutomaticActivity

Android Programs Appendices 21: AutomaticActivity.java package my.last.prog; import java.io.BufferedReader; import java.io.PrintWriter; import java.net.Socket; import my.last.prog.connection.SendMessage; import my.last.prog.listeners.CancelListener; import android.app.Activity; import android.content.pm.ActivityInfo; import android.hardware.Sensor; import android.hardware.SensorEvent; import android.hardware.SensorEventListener; import android.hardware.SensorManager; import android.os.Bundle; import android.os.Handler; import android.util.Log; import android.view.View; import android.view.View.OnClickListener; import android.widget.Button; import android.widget.TextView; import android.widget.Toast; import android.widget.ToggleButton; public class AutomaticActivity extends Activity implements SensorEventListener { protected final String serverIP =/* "192.168.11.9";*/"172.16.9.41"; protected final int serverPort = 8080; protected final int numberOfSonar = 8; public TextView sonarValue0 = null; public TextView sonarValue1 = null; public TextView sonarValue2 = null; public TextView sonarValue3 = null; public TextView sonarValue4 = null; public TextView sonarValue5 = null; public TextView sonarValue6 = null; public TextView sonarValue7 = null;

27

public TextView azimuthView = null; public TextView pitchView = null; public TextView rollView = null; public boolean activated = false; // For Handler public Socket dataTransmition; protected BufferedReader in; public Handler myhandler = new Handler(); public PrintWriter out; protected ToggleButton sensorEnable = null; protected Button cancelButton = null; // These numbers are to get the float public Float numberAzimuth = null; public Float numberPitch = null; public Float numberRoll = null; public SensorManager mySensorManaged; public Sensor mySensor; // Letter to display public String[] myLetters = new String[] { "F", "F+", "F++", "B", "B+", "B++", "L", "L+", "L++", "R", "R+", "R++" }; // My Toast public Toast toast = null; // Test if the value changed private String direction = ""; /** * Called when the activity is first created. */ public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.automatic); setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE); // Initialization sonarValue0 = (TextView) findViewById(R.id.sensorTextView0); sonarValue1 = (TextView) findViewById(R.id.sensorTextView1); sonarValue2 = (TextView) findViewById(R.id.sensorTextView2); sonarValue3 = (TextView) findViewById(R.id.sensorTextView3); sonarValue4 = (TextView) findViewById(R.id.sensorTextView4); sonarValue5 = (TextView) findViewById(R.id.sensorTextView5); sonarValue6 = (TextView) findViewById(R.id.sensorTextView6); sonarValue7 = (TextView) findViewById(R.id.sensorTextView7); azimuthView = (TextView) findViewById(R.id.azimuthView); pitchView = (TextView) findViewById(R.id.pitchView); rollView = (TextView) findViewById(R.id.rollView); cancelButton = (Button) findViewById(R.id.cancelB); sensorEnable = (ToggleButton) findViewById(R.id.sensorToogle); /* try { dataTransmition = new Socket(this.serverIP, this.serverPort); } catch (UnknownHostException e) {

28

Log.e("AutomaticActivity error","UnknownHostException"); } catch (IOException e) { Log.e("AutomaticActivity error","IOException"); } try { out = new PrintWriter(dataTransmition.getOutputStream(),true); } catch (IOException e1) { Log.e("AutomaticActivity","IOException"); }*/ numberAzimuth = Float.parseFloat(azimuthView.getText().toString()); numberPitch = Float.parseFloat(pitchView.getText().toString()); numberRoll = Float.parseFloat(rollView.getText().toString()); // Managed my sensor mySensorManaged = (SensorManager) getSystemService(SENSOR_SERVICE); // When I click on the button sensorEnable.setOnClickListener(new Enable()); cancelButton.setOnClickListener(new CancelListener(this)); //

this.createHandler(); this.disableSensor();

} /** * Create a handler */ public void createHandler(){ new Thread(new Runnable() { public void run() { // TODO Auto-generated method stub myhandler.post( new Runnable() { public void run() { // TODO Auto-generated method stub while(true){ try { // in = new BufferedReader(new InputStreamReader(dataTransmition.getInputStream())); /* for(int i=0;i 6 && (!direction.equalsIgnoreCase("F") && !direction.equalsIgnoreCase("F+") && !direction.equalsIgnoreCase("F++"))) { // if we go forward fast if (numberRoll > 8) { // if we go forward very fast if (numberRoll > 9) { if (!direction.equalsIgnoreCase("F++")) { sendLetter(myLetters[2]); direction = "F++"; return; } } if (!direction.equalsIgnoreCase("F+")) { sendLetter(myLetters[1]); direction = "F+"; return; } } if (!direction.equalsIgnoreCase("F")) { sendLetter(myLetters[0]); direction = "F"; } } } /** * Send the letter to go back */ protected int sendMessageBack() { // if we go back if (numberRoll < 1) { if (numberRoll < -2.5) { sendLetter(myLetters[4]); // 4 = B+ direction = "B+"; this.finish(); // return -1; } if (!direction.equalsIgnoreCase("B")) {

31

sendLetter(myLetters[3]); // 3 = B direction = "B"; return 1; } } return 0; } /** * Send the letter to move on the left */ protected void sendMessageLeft() { // if we go back if (numberPitch < -3 && !direction.equalsIgnoreCase("L")) { sendLetter(myLetters[6]); // 6 = L direction = "L"; } } /** * Send the letter to move on the right */ protected void sendMessageRight() { // if we go back if (numberPitch > 3 && !direction.equalsIgnoreCase("R")) { sendLetter(myLetters[9]); // 9 = R direction = "R"; } }

/*

/* //

/** * Sent the letter to the server * @param letter */ private void sendLetter(String letter) { SendMessage sendMessage = new SendMessage(letter, this.serverIP, this.serverPort+"", this); Thread sendThread = new Thread(sendMessage); sendThread.start();*/ sendMessage.run(); for(int i=0 ;i 0)) { end = false; myServer.stopRobotForAMoment(); while (robot.getSonarRange(5) < 800 && robot.getSonarRange(2) < 800) { if (!end) { myServer.moveForward(robot, "F"); end = true; } } } else if (robot.getSonarRange(4) < 800 && robot.getSonarRange(3) > 800) { end = false; myServer.stopRobotForAMoment(); while (robot.getSonarRange(4) < 800) { if (!end) { myServer.turnLeftRight("L"); end = true; } } myServer.moveForward(robot, "F"); } else if (robot.getSonarRange(3) < 800 && robot.getSonarRange(4) > 800) { end = false; myServer.stopRobotForAMoment(); while (robot.getSonarRange(3) < 800) { if (!end) { myServer.turnLeftRight("R"); end = true; }

54

} myServer.moveForward(robot, "F"); } if ((left < 5 && right < 5)) { myServer.stopRobotForAMoment(); end = false; while (left < 5 && right < 5) { if (!end) { myServer.moveForward(robot, "F"); end = true; } } } else if (left < 10 && right > 10) { myServer.stopRobotForAMoment(); end = false; while (left < 30) { if (!end) { myServer.turnLeftRight("R"); end = true; } left = updateLeft(); } myServer.moveForward(robot, "F"); } else if (right < 10 && left > 10) { myServer.stopRobotForAMoment(); while (right < 30) { if (!end) { myServer.turnLeftRight("L"); end = true; } right = updateRight(); } myServer.moveForward(robot, "F"); } } /** * change the percent of the left part * @return */ public float updateLeft() { // Initialization totalSonarLeft = 0; percentLeft = 0; // Calculate the total of each sensor Right and Left for (int i = 0; i < 4; i++) { totalSonarLeft += robot.getSonarRange(i); } // Calculate the percent of each part percentLeft = (((float) totalSonarLeft / (float) totalOf4Sonar)) * 100; return percentLeft; } /** * change the percent of the right part * @return */

55

public float updateRight() { // Initialization totalSonarRight = 0; percentRight = 0; // Calculate the total of each sensor Right and Left for (int i = 0; i < 4; i++) { totalSonarRight += robot.getSonarRange(i + 4); } // Calculate the percent of each part percentRight = (((float) totalSonarRight / (float) totalOf4Sonar)) * 100; return percentRight; } /** * Stock the value of each sonar into an array */ public void stockValue(){ for (int i = 0; i < 8; i++) { this.sonarValue[i] = robot.getSonarRange(i); } } /** * Return the value of the sonar number i * @param i * @return */ public int getValue(int i){ // return this.sonarValue[i]; return robot.getSonarRange(i); } }

Appendices 33: StockSonarRange.java package Server; import java.io.IOException; import java.io.PrintWriter; import java.net.Socket; import com.mobilerobots.Aria.ArRobot; public class StockSonarRange implements Runnable { public ArRobot robot; public Socket client; private int[] sonarValue; protected PrintWriter out; public StockSonarRange(ArRobot robot, Socket client) { this.robot = robot; this.client = client; this.sonarValue = new int[] { 0, 0, 0, 0, 0, 0, 0, 0 }; } public void run() { while (true) { // try {

56

for (int i = 0; i < 8; i++) { this.sonarValue[i] = robot.getSonarRange(i); } Thread.sleep(30); } catch (InterruptedException e) { }*/ }

/*

} public void getValue(int numberSensor) { try { out = new PrintWriter(this.client.getOutputStream()); out.println(this.sonarValue[numberSensor] + ""); } catch (IOException e) { System.out.println("error stockage"); ; } } }

Appendices 34: ServerSendValues.java package Server; import java.io.BufferedReader; import java.io.IOException; import java.io.InputStreamReader; import java.net.ServerSocket; import java.net.Socket; import com.mobilerobots.Aria.ArRobot; public class ServerSendValues implements Runnable { public static final String SERVERIP = "localhost"; // My system's IP Address public static final int SERVERPORT = 9876; public boolean end; public ArRobot robot; public StockSonarRange test; protected Socket client; public ServerSendValues(ArRobot robot){ this.robot = robot; } public void run() { try { ServerSocket serverSocket = new ServerSocket(SERVERPORT); while (true) { Socket client = serverSocket.accept(); // Accept when a client is coming System.out.println("test"); if(end==false) { //Stock the value of the sensor in an Array of int test = new StockSonarRange(robot, client); Thread stockValue = new Thread(test); stockValue.start(); System.out.println("Stockage launch"); end=true; } try {

57

// Display the message BufferedReader buff = new BufferedReader( new InputStreamReader(client.getInputStream())); String str = buff.readLine(); switch(Integer.parseInt(str)){ case 0 : test.getValue(0); break; case 1 : test.getValue(1); break; case 2 : test.getValue(2); break; case 3 : test.getValue(3); break; case 4 : test.getValue(4); break; case 5 : test.getValue(5); break; case 6 : test.getValue(6); break; case 7 : test.getValue(7); break; } Thread.sleep(30); } catch (Exception e) { System.out.println("S: Error"); } finally { client.close(); } } } catch (IOException e) { System.out.println("client Send error"); } } }

Appendices 35: R.java (Generate by eclipse) /* AUTO-GENERATED FILE. DO NOT MODIFY. * * This class was automatically generated by the * aapt tool from the resource data it found. It * should not be modified by hand. */ package my.last.prog; public final class R { public static final class attr { } public static final class drawable { public static final int ic_launcher=0x7f020000; } public static final class id { public static final int AbsoluteLayout1=0x7f050011; public static final int AzimuthTextV=0x7f050007; public static final int LinearLayout=0x7f050013; public static final int PitchTextV=0x7f050006; public static final int RollTextV=0x7f050008; public static final int Title=0x7f050003; public static final int automaticButton=0x7f050017; public static final int azimuthView=0x7f050004; public static final int background=0x7f05002c; public static final int button1Cancel=0x7f050012; public static final int buttonCancel=0x7f05002b;

58

public static final int buttonCancel2=0x7f05002d; public static final int buttonConnect=0x7f050019; public static final int buttonLabs=0x7f050029; public static final int cancelB=0x7f05000c; public static final int cancelButton=0x7f05001a; public static final int copyright=0x7f05001b; public static final int defaultPort=0x7f050026; public static final int displayButton=0x7f050016; public static final int downButton=0x7f05001f; public static final int fasterButton=0x7f050023; public static final int hachinohe=0x7f05001c; public static final int icon=0x7f050014; public static final int ipTextView=0x7f050027; public static final int leftButton=0x7f05001d; public static final int manualButton=0x7f050018; public static final int messageEditText=0x7f05002a; public static final int numberPort=0x7f050024; public static final int pitchView=0x7f050005; public static final int rightButton=0x7f05001e; public static final int rollView=0x7f050009; public static final int sendButton=0x7f050015; public static final int sensorTextView0=0x7f050000; public static final int sensorTextView1=0x7f050001; public static final int sensorTextView2=0x7f05000a; public static final int sensorTextView3=0x7f050002; public static final int sensorTextView4=0x7f05000d; public static final int sensorTextView5=0x7f05000e; public static final int sensorTextView6=0x7f05000f; public static final int sensorTextView7=0x7f050010; public static final int sensorToogle=0x7f05000b; public static final int serverIP=0x7f050028; public static final int serverPort=0x7f050025; public static final int stopButton=0x7f050020; public static final int toggleButtonSpeed=0x7f050021; public static final int upButton=0x7f050022; } public static final class layout { public static final int automatic=0x7f030000; public static final int displaysensor=0x7f030001; public static final int main=0x7f030002; public static final int manual=0x7f030003; public static final int sendmessages=0x7f030004; public static final int testconnect=0x7f030005; } public static final class string { public static final int SendButton=0x7f040009; /** Used to send a message to the server */ public static final int SendText=0x7f040008; /** First Activity */ public static final int app_name=0x7f040000; public static final int automaticControl=0x7f040007; public static final int azimuthText=0x7f040014; public static final int azimuthView=0x7f040011; /** For all activity */ public static final int cancelButton=0x7f04001e; public static final int copyright=0x7f040001; public static final int defaultPort=0x7f04000f;

59

public static final int displayButton=0x7f040004; public static final int downButton=0x7f040018; public static final int faster=0x7f04001d; public static final int hachinohe=0x7f040002; public static final int laboratory=0x7f04000b; public static final int leftButton=0x7f040019; public static final int manualControl=0x7f040005; public static final int nothing=0x7f04001f; public static final int numberIP=0x7f04000c; /** Used to display each values */ public static final int numberOfSensor=0x7f040010; public static final int numberPort=0x7f04000d; public static final int pitchText=0x7f040015; public static final int pitchView=0x7f040012; public static final int rightButton=0x7f04001a; public static final int rollText=0x7f040016; public static final int rollView=0x7f040013; public static final int sendMessage=0x7f040003; /** Test display */ public static final int sensor0=0x7f040020; public static final int sensor1=0x7f040021; public static final int sensor2=0x7f040022; public static final int sensor3=0x7f040023; public static final int sensor4=0x7f040024; public static final int sensor5=0x7f040025; public static final int sensor6=0x7f040026; public static final int sensor7=0x7f040027; public static final int serverIP=0x7f04000a; public static final int serverPort=0x7f04000e; public static final int speedButton=0x7f04001c; public static final int stopButton=0x7f04001b; /** Test connection */ public static final int testConnect=0x7f040028; public static final int testConnection=0x7f040006; /** Directions for manual */ public static final int upButton=0x7f040017; } }

60

Android XML: Appendices 36: AndroidManifest.xml

61

Appendices 37: string.xml Robot Control v1.95 created by CHA Jerome 八戸工業高等専門学校 Send messages to server Display values of the sensors Manual Test Connection Automatic Enter a message Send IP of server Laboratory IP : Port : Port of server Default Values of the Sensors : 0.0 0.0 0.0 Azimuth : Pitch : Roll : F B L R Disconnect Speed F+ Exit 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Test connection

62

Appendices 38: automatic.xml

Appendices 39: displaysensor.xml

66

Appendices 40: main.xml

67



Appendices 41: manual.xml

Appendices 42: sendmessage.xml

70

Appendices 43: testconnect.xml

71



72

Laboratory Appendices 44: Laboratory room.

Appendices 45: Robots in the laboratory

73

Others: Appendices 46: Kabushima Shrine

Appendices 47: Japaneses Umineko (海猫: cats of the sea)

74

Appendices 48: Server multiclients

Appendices 49: Robot and its parts

Appendices 50: All movements

75

Glossary Activity: In Android programming, this term is like a JFrame in Java. We can draw and add button by a simple Drag and Drop or by using xml. AI: (Artificial Intelligence) this intelligence is created by a human to a robot or machine. They can react like a truth person. Android: it is a Linux-based operating system for mobile devices such as smartphones and tablet computers. API: Application Programming Interface: specification intended to be used as an interface by software components to communicate with each other. Aria: (Advanced Robotics Interface for Applications) it was written in C++ language. It is client-side software for easy, high-performance access to and management of the robot Boolean: it refers to represents either true or false, that name comes to George Boole. Cherry blossom: very famous in japan. The blooming of flowers is in spring, and Japanese people usually go under the trees to eat together. Cast: In the Java programming language, the casting/coercing operation is used to convert between types and the instanceof operator is used to check for type information at run time. ID: Identification number IUT: it means “Institut Universitaire de Technologie” in France. MVC: (Model View Constructor) it is a powerful pattern which can solve any problems with its three parts. OS: (Operating system) it is a set of software that manages computer hardware resources and provides common services for computer programs. SDK: System Development Kit: is typically a set of software development tools that allows for the creation of applications for certain software. Shinkansen: King of speed train, like French’s TGV but speeder. In Japan, to take this train it is very expensive. Socket: It’s an endpoint of an inter-process communication flow across a computer network. TCP/IP: (Transmission Control Protocol) this protocol is in the layer “Transport” of OSI layers. Thread: A Thread is a concurrent unit of execution. It has its own call stack for methods being invoked, their arguments and local variables.

76

ToogleButton: it is a king of button in Android that when we click on it, it will be activate. Another click will disable it. Towada Lake: a beautiful lake surrounding by mountains. UDP: (User Datagram Protocol) One of the core members of the internet protocol. It works in the disconnected mode. XML: Extensible Markup Language: it is a markup that defines a set of rules for encoding documents in a format that a human and a pc can read.

77

Bibliography http://stackoverflow.com/ This website is a kind of forum where some people need help about any error they have in Java programming language. It was very useful when we have the same error. http://www.siteduzero.com/tutoriel-3-10601-apprenez-a-programmer-en-java.html This website speaks about how to program in Java. It was very interesting for socket programming. http://fr.wikipedia.org/wiki/Wikip%C3%A9dia:Accueil_principal Everyone knows Wikipedia when we need to do a research and get more information about something. I used it when I need to have some information about Hachinohe for example. http://queinnec.perso.enseeiht.fr/Ens/Chat/socket-java.html I usually go to this website when I have some error with a socket. http://www.ai.rug.nl/vakinformatie/pas/content/Aria-manual/main.html It is the best website to know more about Aria’s libraries. It is with C++ programming language but we need to go on this website once time if we use Aria. http://developer.android.com/reference/android/app/Activity.html This is the Android’s documentation. We can find all Android’s class and implementations. There are many samples programs too. http://docs.oracle.com/javase/7/docs/api/ This is the Java’s documentation with the version 7 of Java. We can find any class and a lot of samples programs on this website. http://robots.mobilerobots.com/MobileSim/README.html If you want to install MobileSim on your laptop you need to go there. There are all versions (but don’t forget, it cannot be used with Windows 64 bits). http://robots.mobilerobots.com/wiki/ARIA On this website, you can get all Aria’s libraries for free.

78