Virtual Reality in Manufacturing

CONTENT

•  INTRODUCTION
•  VIRTUAL REALITY
•  FLEXIBLE MANUFACTURING
•  BENEFITS OF VIRTUAL REALITY IN FLEXIBLE MANUFACTURING
•  IMPLEMENTATION OF A REMOTE VIRTUAL REALITY SYSTEM
•  CONCLUSIONS, FUTURE PLANS
•  REFERENCES

1. INTRODUCTION

In some cases the users or analysts are sitting in another country or continent, that make sense of remote application of virtual reality based simulation.
In the following sections a brief introduction will be given to both Virtual Reality (VR) and Flexible Manufacturing Systems (FMS), then the benefits of VR in FMS will be discussed, and finally an implementation of a remote virtual reality system will be introduced.

2. VIRTUAL REALITY

The first step in immersion is to provide the spatial relationship between the user and the environment. This may be achieved through the use of location and orientation tracking devices. Tracking the user’s head allows the computer to generate correct images for the viewer. Tracking other parts of the body allows further interaction of user with the environment.

The typical VR display is a wall-projection display that uses either a large computer screen or a rear projection display. These systems can provide a good sense of depth but the feeling of immersion is very low. They are relatively inexpensive and non-intrusive. A greater impression of immersion may be achieved through head-mounted or helmet-mounted displays (HMD), which consist of a pair of LCD or CRT display devices, mounted on a helmet worn by the user. In some HMD there is only one display that is connected to a system of lenses and mirrors that conveys the image into the retina of the eyes.

3. FLEXIBLE MANUFACTURING

The planning, design and operating of FMS is a very complex task, because of the high value of the system, the complexity of the control algorithms, the integrated structure of the controlling components of the system, and the large amounts of real-time data on machine and product status.

4. BENEFITS OF VIRTUAL REALITY IN FLEXIBLE MANUFACTURING

Manufacturing basically comprises four classes of engineering tasks: product design, production design, implementation of production system, and operation of the actual production system. These tasks could benefit from the application of VR through the high-fidelity visualization, providing model and situation representation in the 3D space, and the power of interactive handling of large data sets in real-time and near real-time.

For applying VR technology in manufacturing systems new type of virtual interfaces should be applied as the evolution of the man-machine interface from the traditional devices (computer screen, keyboard, mouse) to the more sophisticated tools, like head-mounted display, data gloves, flight simulator joystick, etc.

FACTORY LAYOUT AND ERGONOMICS

PROCESS PLANNING

MACHINING SIMULATION

CONTROL OF FMS

When the VR technology is integrated with high-bandwidth network channels, those environments can offer tele-presence features [6], which support experts to share remotely the task of a local operator. These situations are of very high value, for trained experts can virtually be present at any spot around the globe. The tele-presence feature can help in tracking real manufacturing processes, where the system and the observer are in different locations. Tele-presence can be extended to tele-operation, when the system is not only observed, but controlled from a remote location. In both cases VR technology can be useful when it is impossible to establish an appropriate feedback from the remote site to the observer or controller. In these situations the real system should be modeled and often visualized, so that the remote observer or controller gets information not only from the real operation, but also from the model that integrates real and simulated data.

Regarding technology aspects, FMS environments are dangerous machine structures, and operators are not allowed to roam around freely in the cells. For example Automated Guided Vehicles (AGV) or robotic cells ought not to operate, when humans are present in their working environment. The VR technology could allow the operators to run the FMS, and only be virtually present at hazardous sites. Such applications suggest very sophisticated, real-time multi-media database tools, as well.

5. IMPLEMENTATION OF A REMOTE VIRTUAL REALITY SYSTEM

THE SIMULATED PILOT PLANT

An UNIMATE PUMA 760 assembly robot can be found in the assembly cell, which can serve two pallets at the same time. In the measuring cell an OPTON UMC 850 measuring unit operates. Our VR system concentrates on the simulation of the manufacturing and the assembly cell of this plant. Our VR system concentrates on the simulation of the manufacturing and the assembly cell of this plant.

TOOLS AND PREPARATION OF THE VR SYSTEM

The PC Glasstron is a unique head mounted display that creates a high resolution, virtual 30" image when connected to the computer (or video source). With built-in ear buds for stereo sound it has full multimedia capability. Its internal dual LCD panels create an impressive, large screen. This tool has been used successfully for virtual round table scenarios at GMD [7].

For navigation we have chosen a flight simulator joystick by QuickShot (see Fig.3.) with which the user can drive a virtual helicopter that flies around the simulated objects.

Creating Virtual Reality environment for manufacturing applications requires powerful simulation and animation software tools, as well. Taylor ED (Enterprise Dynamics) is a family of software products to let you model, simulate, visualize and control business and real-life processes [8]. It offers the necessary features to build complicated models and animations of manufacturing systems, as well. Due to the open structure of the program, the number of applications is unlimited. Different suppliers will continue to publish new libraries of atoms (the building blocks in ED) to be used with Taylor ED. One or more 2D and 3D views may be defined in Taylor ED. Each view may have a different viewpoint, camera position and camera angle. Taylor ED is an effective simulation tool, but the quality of its 3D images is not satisfactory for our purposes.

For this reason we decided to use a VRML tool for visualization controlled by Taylor ED, which provides flexibility and generality. Several tools exist to convert almost any CAD format to VRML, enabling geometric models to be easily imported from whatever format they already exist in without having to redraw these. VRML 3D browsers can be installed as free plug-in within common free 2D browsers, such as Netscape or Internet Explorer. Furthermore, as VRML can exchange messages with external programs written in Java, scenes can be enhanced with Java’s powerful computational and network facilities. For instance, positional data can be received via sockets either from the web or from other programs on the local computer, interpreted and treated in Java and then finally routed to the VRML scene.

We have created the VRML images of the simulated pilot plant using 3D Studio MAX, extended with real JPG images. The JPG images have been created by a Panasonic digital video camera, and were imported as textures to make the appearance of the simulation objects more realistic. We used Cortona VRML plug-in in our project because it is small, fast and rich in important features as follow. VRML objects can be embedded in HTML pages and can be viewed full screen, as well. It supports input devices (e.g. joystick) connected to the game port, as well as real 3D view which is available with special hardware elements (appropriate VGA card with glasses and a monitor with fast refresh rate).

Since our VRML scene should work in a remote controlled way, we had to choose one from the several possibilities that the VRML standard offers to program the virtual world. For technical reasons the External Authoring Interface (EAI) has been chosen. With the EAI a Java applet can get reference to the objects of the VRML scene and modify their attributes to generate the necessary changes in the appearance of the virtual world.

REMOTE OPERATION

The Simulator itself has been realized in 4Dscript that is the programming language of Taylor ED. It gives the possibility to establish network connections via sockets, so every atom that has to be visualized can send their state variables through the socket connection. If the state of an atom (position, orientation, color, etc.) changes, then the atom sends this new information towards a Java application, called Transmitter. During continuous state changing (e.g.: a moving AGV) an event is generated periodically in the atom to send the state variables, which needed to generate the animation of the object’s movement.

The Transmitter is a Java application that receives the object’s updated state information from the simulation in Taylor ED and sends it toward the Receiver, that is an applet controlling the scene in the VR Browser. The work of the Transmitter starts with a main thread. After some initialization the main thread stops, waiting for the Simulator to connect. Accepting the connection request of the simulation, a new thread is created called ServeTED to receive and store the arriving information from the Simulator. After creating ServeTED the main thread blocks again, waiting for the applets to make their requests. As soon as an applet wants to connect, a new thread is created to serve it (ServeApplet), and the main thread waits for another applet’s request. A thread, called ServeApplet begins to run when its applet makes a request for new data, and sends the state variables all of the objects which has been modified since the last request of the applet.

There are two main reasons that justify the use of this middle tier application in the chain of the communication. First of all, it gives more control in the task of serving multiple clients as the 4Dscript could give, and it enables the usage of different socket types (datagaram instead of stream, see later), if we have to change it in the further development of this project. Another benefit is that the Transmitter and the Web Server (which hosts the visualizing HTML page) could run on the same machine, and in this case, we don’t have to use signed applet. The applets are allowed opening network connection towards the machine from which they have been downloaded without any special security permission.

There are restrictions applied to applets because for security reasons. For example they can’t read or write files on the client computer or can’t open a socket towards a different computer from they have been downloaded. Although by signing an applet these restriction can be lifted. To sign an applet a digital certificate is needed from a Certificate Authority (CA). With the help of this certificate the applets can be signed. A signed applet displays a message when the browser downloads it to inform the user who the signer of the applet is, and from which CA originates the certificate. If the user trusts both the signer and the CA he or she will choose to launch the applet.

The user can immerse in the 3D animation (see Fig.5.) through a VRML page available on the Internet (www.sztaki.hu/~sarkozy/vrm/cella.wrl ) using IE 5.0 or later that downloads the Cortona VRML plug-in if it is not already installed. The virtual world appears either in the head-mounted display or in a graphical display. The user can navigate in the system like she or he were sitting on a helicopter using the flight simulator joystick. It is possible to go close and far, up and down, right and left to the objects. The user can turn, and go around as she or he really was in the system.

After the HTML page has been downloaded, the user can examine the virtual manufacturing cell in the VRML browser, and the applet offers the possibility to connect the scene to the simulation.

The applet, like the Transmitter is multi threaded, too. While one thread is dealing with the communication and updating the VRML scene, the other monitoring the user’s interactions. It is planned to enable the user to interact with the simulation either through the VRML or the user interface of the applet. It is essential to let the user at least to tune the simulation speed, stop or reset it.

6. CONCLUSIONS, FUTURE PLANS

During development we have realized, that the speed of the communication can be critical via the network using TCP/IP. Instead of TCP the User Datagram Protocol (UDP) can be another solution, which allows for rapid, connection less communication over the Internet. UDP is fast as it contains very little flow control, unlike TCP that is used by applications such as Telnet and FTP. The price for the heavily reduced communication overhead is that packages occasionally can be lost when UDP is used. When simulations are run on reliable networks or on single hosts, this is rarely a problem. But when simulations are run over the Internet, package loss must be expected and the protocol using UDP (in this case DIS) must tolerate this. This concept has been applied in EURODOCKER project [11]
We are also planning to use EM-Plant for Simulator, which is the 3D version of the well-known simulation system Simple++ [12].

7. REFERENCES

Copyright © 2001 by Sándor Kopácsi <kopacsi@sztaki.hu>

Research Project in MTA SzTAKI at CIM Research Laboratory