What is Teleoperation?

‍

Definition of Teleoperation

Teleoperation, also known as remote robot control, is the process of operating a robotic system from a distance using wired or wireless communication. Unlike fully autonomous robots, which operate based on pre-programmed logic and artificial intelligence, teleoperated robots rely on human input to execute commands in real time. This enables precise control and intervention, making it a valuable technology for a wide range of applications.

How Teleoperation Works

Teleoperation consists of three key components:

1. User Interface (UI): The control medium that allows operators to send commands. This can be a mobile application, a keyboard interface, a joystick, VR controllers, or even EEG (brain-computer interface) systems.

2. Communication System: The network through which control commands are sent to the robot and feedback is received. This can be WiFi, Bluetooth, radio waves, satellite communication, or cellular networks.

3. Robotic System: The hardware and software components that execute commands and provide movement or actions. This includes sensors, actuators, controllers, safety systems, and feedback mechanisms.

‍

Why is Teleoperation Important?

Teleoperation is essential in robotics because it enables human intervention in situations where automation alone is not sufficient. Some key benefits include:

- Remote Accessibility: Operators can control robots from different locations, reducing the need for physical presence in dangerous or inaccessible areas.

- Enhanced Safety: Reduces human exposure to hazardous environments such as high-radiation zones, deep-sea exploration, and outer space.

- Precision Control: Provides real-time human decision-making and adjustments, leading to higher accuracy than fully automated systems.

- Scalability: A single operator can control multiple robots simultaneously, improving efficiency in industrial and research settings.

- Versatility: Teleoperation can be applied across various fields, including healthcare, agriculture, defense, logistics, and research.

Applications of Teleoperation

Teleoperation has been successfully implemented in various industries, proving its flexibility and effectiveness:

- Industrial Robotics: Factories use teleoperated robotic arms for assembly, welding, material handling, and quality control.

- Autonomous Vehicles: Warehouse robots, delivery drones, and self-driving forklifts can be remotely managed.

- Medical Robotics: Remote-controlled robotic arms perform telesurgery, enabling doctors to operate on patients from thousands of kilometers away.

- Defense and Security: Bomb disposal robots, surveillance drones, and robotic reconnaissance units help security forces in high-risk areas.

- Space Exploration: NASA’s rovers on Mars, such as Perseverance, are controlled remotely from Earth to explore and analyze planetary surfaces.

- Search and Rescue: Teleoperated robots navigate collapsed buildings and hazardous environments to locate and assist survivors.

- Agriculture and Farming: Remote-controlled tractors, robotic planters, and harvesting machines optimize agricultural productivity.

- Hazardous Material Handling: Robots manage toxic chemicals and nuclear waste, reducing human risk exposure.

Methods for Teleoperating Acrome SMD Robots

Acrome SMD systems provide a robust platform for teleoperation. Depending on the application, different control methods can be used:

Teleoperation via Mobile Application: Using an Android APK for wireless robot control, allowing for flexibility and ease of use.

Teleoperation via PC Keyboard: Utilizing keyboard commands to control robots, ideal for manual operation and research.

Teleoperation via Joystick or Game Controller: Offering an intuitive control method for applications requiring precise movements.

Teleoperation Using a Mobile Application (APK)

Why Use a Mobile Application?

A mobile application provides an easy-to-use, wireless control system for Acrome SMD-powered robots. Advantages include:

- Wireless Connectivity: Eliminates the need for physical cables, enhancing mobility and range.

- User-Friendly Interface: Touchscreen interfaces provide intuitive control, reducing the learning curve for new operators.

- Remote Access: Allows operators to control robots from different locations via an internet connection.

- Real-Time Responsiveness: Ensures instant feedback and execution of commands, providing a seamless experience.

- Multi-Device Support: Multiple users can control or monitor robots from different devices simultaneously.

‍

‍

System Architecture

1. User Input: The operator interacts with the mobile app to send movement commands.

2. Wireless Communication: The app transmits control signals to a Flask API running on a Raspberry Pi via WiFi.

3. Command Processing: The Flask API processes the received signals and transmits them to the Acrome SMD motor controllers.

4. Motor Execution: Acrome SMD controllers adjust motor speeds and directions accordingly.

5. Safety Mechanism: A monitoring system continuously checks motor performance and stops the robot if any issues arise.

Python Code for Mobile Application-Controlled Robot Movement:

1from flask import Flask, request
2from smd.red import *
3from serial.tools.list_ports import comports
4from platform import system
5import threading
6import time
7
8app = Flask(__name__)
9class PIDController:
10    def __init__(self, kp, ki, kd):
11        self.kp = kp
12        self.ki = ki
13        self.kd = kd
14        self.previous_error = 0
15        self.integral = 0
16
17    def calculate(self, error, delta_time):
18        self.integral += error * delta_time
19        derivative = (error - self.previous_error) / delta_time if delta_time > 0 else 0
20        output = (self.kp * error) + (self.ki * self.integral) + (self.kd * derivative)
21        self.previous_error = error
22        return max(min(output, 100), -100)  # Clamp output to motor speed range
23# Robot Setup
24def USB_Port():
25    ports = list(comports())
26    usb_names = {
27        "Windows": ["USB Serial Port"],
28        "Linux": ["/dev/ttyUSB"],
29        "Darwin": [
30            "/dev/tty.usbserial",
31            "/dev/tty.usbmodem",
32            "/dev/tty.SLAB_USBtoUART",
33            "/dev/tty.wchusbserial",
34            "/dev/cu.usbserial",
35            "/dev/cu.usbmodem",
36            "/dev/cu.SLAB_USBtoUART",
37            "/dev/cu.wchusbserial",
38        ],
39    }
40    os_name = system()
41    if ports:
42        for port, desc, hwid in sorted(ports):
43            if any(name in port or name in desc for name in usb_names.get(os_name, [])):
44                return port
45        print("Current ports:")
46        for port, desc, hwid in ports:
47            print(f"Port: {port}, Description: {desc}, Hardware ID: {hwid}")
48    else:
49        print("No port found")
50    return None
51
52port = USB_Port()
53smd = Master(port) if port else None
54if smd:
55    smd.attach(Red(0))  # Left motor
56    smd.attach(Red(1))  # Right motor
57    smd.set_operation_mode(0, OperationMode.PWM)
58    smd.set_operation_mode(1, OperationMode.PWM)
59    smd.set_shaft_rpm(0, 100)
60    smd.set_shaft_rpm(1, 100)
61    smd.set_shaft_cpr(0, 6533)
62    smd.set_shaft_cpr(1, 6533)
63    smd.enable_torque(0, 1)
64    smd.enable_torque(1, 1)
65    left_pid = PIDController(kp=23.55, ki=0.00, kd=18.65)
66    right_pid = PIDController(kp=21.37, ki=0.00, kd=18.15)
67    base_speed = 60 
68    turning_speed = 40
69def stop_robot():
70    """Stop the robot by setting duty cycle to 0."""
71    smd.set_duty_cycle(0, 0)
72    smd.set_duty_cycle(1, 0)
73
74def watchdog_check():
75    """Check periodically if the motors are stuck or not working."""
76    while True:
77        # Logic to check if motors are stuck (e.g., if duty cycle hasn't changed for a while)
78        # You can implement a simple check based on time or position feedback.
79        # For now, we just print a simple message.
80        print("Watchdog: Checking motor status.")
81        time.sleep(5)  # Periodically check every 5 seconds
82
83# Start a separate thread for watchdog monitoring
84watchdog_thread = threading.Thread(target=watchdog_check, daemon=True)
85watchdog_thread.start()
86
87@app.route('/control', methods=['POST'])
88def control():
89    global last_command_time
90    data = request.get_json()
91    direction = data.get('direction', '')
92    target_speed = 60  # Hedef hız
93    turning_speed = 40  # Dönüş sırasında kullanılacak hız
94    left_speed = 0
95    right_speed = 0
96
97    if smd:
98        last_command_time = time.time()  
99        current_time = time.time()
100        delta_time = current_time - last_command_time
101
102        if direction == '1':  # Move forward
103            error = target_speed
104            left_speed = left_pid.calculate(error, delta_time)
105            right_speed = right_pid.calculate(error, delta_time)
106            smd.set_duty_cycle(0, -left_speed)  # Left motor forward
107            smd.set_duty_cycle(1, right_speed)
108        elif direction == '4':  # Move backward
109            error = -target_speed
110            left_speed = left_pid.calculate(error, delta_time)
111            right_speed = right_pid.calculate(error, delta_time)
112            smd.set_duty_cycle(0, -left_speed)  # Left motor backward
113            smd.set_duty_cycle(1, right_speed)  # Right motor backward
114        elif direction == '3':  # Turn right
115           smd.set_duty_cycle(0, -turning_speed)  # Left motor forward
116           smd.set_duty_cycle(1, 0)
117
118        elif direction == '2':  # Turn left
119           smd.set_duty_cycle(0, 0)  
120           smd.set_duty_cycle(1, turning_speed)
121
122        elif direction == '0':  # Stop
123            stop_robot()
124            return {"status": "success", "direction": direction}
125
126
127    return {"status": "error", "message": "Robot not connected"}
128
129if __name__ == '__main__':
130    app.run(host='0.0.0.0', port=5005, debug=True)

‍

Teleoperation via PC Keyboard

Why Use a Keyboard for Teleoperation?

A PC keyboard can be used to send commands to Acrome SMD robots, providing a simple and effective way to control movements. This method is ideal for research, industrial applications, and testing environments where precise control is required.

Advantages of Keyboard-Based Teleoperation

- Low Latency Execution: Keyboard input ensures immediate response to commands.

- High Precision Control: Useful for manual operation requiring fine movement adjustments.

- Reliable Performance: No dependency on wireless connections, ensuring uninterrupted operation.

- Easy Integration: Simple to implement using Python and other programming languages.

Python Code for Keyboard-Controlled Robot Movement:

1from pynput import keyboard
2import time
3from smd.red import *
4from serial.tools.list_ports import comports
5from platform import system
6class PIDController:
7    def __init__(self, kp, ki, kd):
8        self.kp = kp
9        self.ki = ki
10        self.kd = kd
11        self.previous_error = 0
12        self.integral = 0
13
14    def calculate(self, error, delta_time):
15        self.integral += error * delta_time
16        derivative = (error - self.previous_error) / delta_time if delta_time > 0 else 0
17        output = (self.kp * error) + (self.ki * self.integral) + (self.kd * derivative)
18        self.previous_error = error
19        return max(min(output, 100), -100)  
20    
21    
22def USB_Port():
23    ports = list(comports())
24    usb_names = {
25        "Windows": ["USB Serial Port"],
26        "Linux": ["/dev/ttyUSB"],
27        "Darwin": [
28            "/dev/tty.usbserial",
29            "/dev/tty.usbmodem",
30            "/dev/tty.SLAB_USBtoUART",
31            "/dev/tty.wchusbserial",
32            "/dev/cu.usbserial",
33        ]
34    }
35    os_name = system()
36    if ports:
37        for port, desc, hwid in sorted(ports):
38            if any(name in port or name in desc for name in usb_names.get(os_name, [])):
39                print("Connected!")
40                return port
41        print("Available ports:")
42        for port, desc, hwid in ports:
43            print(f"Port: {port}, Description: {desc}, HWID: {hwid}")
44    else:
45        print("No ports detected!")
46    return None
47
48
49def teleoperate_smd():
50    print("Use W/A/S/D to control the robot. Press Q to quit.")
51
52    port = USB_Port()
53    if not port:
54        print("No suitable port found. Exiting...")
55        return
56
57    try:
58        smd = Master(port)
59        smd.attach(Red(0))  # Left motor (ID 0)
60        smd.attach(Red(1))  # Right motor (ID 1)
61
62        smd.enable_torque(0, 1)
63        smd.enable_torque(1, 1)
64
65        left_pid = PIDController(kp=24.96, ki=0.00, kd=19.10)
66        right_pid = PIDController(kp=46.97, ki=0.00, kd=18.96)
67
68        base_speed = 100  
69        turning_speed = 100  # Speed for turning
70        last_time = time.time()
71
72        def on_press(key):
73            nonlocal last_time
74
75            try:
76                # Calculate time difference
77                current_time = time.time()
78                delta_time = current_time - last_time
79                last_time = current_time
80
81                if hasattr(key, 'char'):
82                    if key.char == 'w':  # Move forward
83                        print("Move Forward")
84                        error = base_speed
85                        left_speed = left_pid.calculate(error, delta_time)
86                        right_speed = right_pid.calculate(error, delta_time)
87                        # Send commands to both motors simultaneously
88                        smd.set_duty_cycle(0, -left_speed)  # Left motor forward
89                        smd.set_duty_cycle(1, right_speed)  # Right motor forward
90                    elif key.char == 's':  # Move backward
91                        print("Move Backward")
92                        error = -base_speed
93                        left_speed = left_pid.calculate(error, delta_time)
94                        right_speed = right_pid.calculate(error, delta_time)
95                        # Send commands to both motors simultaneously
96                        smd.set_duty_cycle(0, -left_speed)  # Left motor backward
97                        smd.set_duty_cycle(1, right_speed)  # Right motor backward
98                    elif key.char == 'a':  # Turn left
99                        print("Turn Left")
100                        # Send commands to both motors simultaneously
101                        smd.set_duty_cycle(0, 0)  # Left motor stopped
102                        smd.set_duty_cycle(1, turning_speed)  # Right motor forward
103                    elif key.char == 'd':  # Turn right
104                        print("Turn Right")
105                        # Send commands to both motors simultaneously
106                        smd.set_duty_cycle(0, -turning_speed)  # Left motor forward
107                        smd.set_duty_cycle(1, 0)  # Right motor stopped
108                    elif key.char == 'q':  # Quit
109                        print("Exiting...")
110                        return False
111            except AttributeError:
112                pass
113
114        def on_release(key):
115            # Stop both motors simultaneously
116            smd.set_duty_cycle(0, 0)
117            smd.set_duty_cycle(1, 0)
118
119        # Start keyboard listener
120        with keyboard.Listener(on_press=on_press, on_release=on_release) as listener:
121            listener.join()
122
123    except Exception as e:
124        print(f"Error: {e}")
125    finally:
126        # Stop both motors simultaneously during cleanup
127        smd.set_duty_cycle(0, 0)
128        smd.set_duty_cycle(1, 0)
129        smd.enable_torque(0, 0)
130        smd.enable_torque(1, 0)
131        smd.close()
132        print("SMD connection closed.")
133
134teleoperate_smd()

‍

Teleoperation via Joystick or Game Controller

‍

Why Use a Joystick or Game Controller?

Using a joystick or game controller for teleoperation provides a more intuitive and precise control method, especially for applications requiring real-time movement and fine adjustments. Compared to keyboard-based control, joysticks offer smooth transitions, making them ideal for robot navigation, industrial automation, and gaming-related robotics projects

‍

Advantages of Joystick-Based Teleoperation

• Analog Control: Unlike keyboards with discrete inputs, joysticks provide a range of motion, enabling gradual speed and direction adjustments.

• Ergonomic Design: Game controllers are comfortable and easy to use, reducing fatigue for long teleoperation sessions.

• Dual-Axis Movement: The ability to control both X and Y axes simultaneously enhances maneuverability.

• Multi-Button Functions: Additional buttons can be assigned for robotic arm control, mode switching, or emergency stops.

• Wireless Options: Many joysticks and game controllers support Bluetooth or RF connections, allowing remote control without physical constraints.

‍

System Architecture

1. User Input: The operator uses a joystick or game controller to provide movement commands.

2. Signal Processing: The joystick transmits X and Y axis values, and button presses are detected for additional actions.

3. Communication System: Signals are transmitted to the Acrome SMD controller via USB, Bluetooth, or WiFi.

4. Motor Execution: The Acrome SMD motor controllers adjust motor speeds and directions based on joystick input.

5. Safety Mechanism: The system monitors joystick drift and connection stability, ensuring smooth and safe operation.

Python Code for Joystick-Controlled Robot Movement:

1import time
2from smd.red import Master, Red
3from serial.tools.list_ports import comports
4from platform import system
5
6# Detect USB Port
7def detect_usb_port():
8    """Detects the connected USB port for communication."""
9    ports = list(comports())
10    usb_names = {
11        "Windows": ["USB Serial Port"],
12        "Linux": ["/dev/ttyUSB"],
13        "Darwin": [
14            "/dev/tty.usbserial", "/dev/tty.usbmodem",
15            "/dev/tty.SLAB_USBtoUART", "/dev/tty.wchusbserial",
16            "/dev/cu.usbserial", "/dev/cu.usbmodem",
17            "/dev/cu.SLAB_USBtoUART", "/dev/cu.wchusbserial",
18        ]
19    }
20    os_name = system()
21    if ports:
22        for port in ports:
23            if any(name in port.device or name in port.description for name in usb_names.get(os_name, [])):
24                return port.device
25    return None
26
27# Initialize USB connection
28serial_port = detect_usb_port()
29if not serial_port:
30    print("No valid USB port detected. Check the connection.")
31    exit()
32
33# Initialize SMD Red Motors
34master = Master(serial_port)
35
36MOTOR_LEFT_ID = 0   # Left Motor
37MOTOR_RIGHT_ID = 1  # Right Motor
38JOYSTICK_ID = 5     # Joystick module ID
39
40motor_left = master.attach(Red(MOTOR_LEFT_ID))   # Attach Left Motor
41motor_right = master.attach(Red(MOTOR_RIGHT_ID)) # Attach Right Motor
42
43# Set motors to PWM mode and enable torque
44master.set_operation_mode(MOTOR_LEFT_ID, 0)  
45master.set_operation_mode(MOTOR_RIGHT_ID, 0)  
46master.enable_torque(MOTOR_LEFT_ID, True)
47master.enable_torque(MOTOR_RIGHT_ID, True)
48
49# Define Dead Zone for Joystick
50DEAD_ZONE = 15  # Ignore small movements
51previous_state = None  # Store the last printed joystick state
52
53# Motor Control with Joystick (Fixed Four Directions, Corrected Motor Directions)
54def control_motors():
55    global previous_state
56
57    print("Motor control via joystick is active...")
58    print("Waiting for joystick movement...")
59
60    while True:
61        joystick = master.get_joystick(MOTOR_LEFT_ID, JOYSTICK_ID)
62        if joystick is None:
63            print("Joystick data could not be read. Check the connection.")
64            time.sleep(0.1)
65            continue
66
67        x_axis, y_axis, button_pressed = joystick[0], joystick[1], joystick[2]
68
69        # Ignore small movements (Dead Zone)
70        if abs(x_axis) < DEAD_ZONE:
71            x_axis = 0
72        if abs(y_axis) < DEAD_ZONE:
73            y_axis = 0
74
75        # Determine movement direction (Only 4 Directions)
76        movement = "Stopped"
77        left_motor_speed = 0
78        right_motor_speed = 0
79
80        if y_axis > 0:  # Forward
81            movement = "Moving Forward"
82            left_motor_speed = -100
83            right_motor_speed = 100
84        elif y_axis < 0:  # Backward
85            movement = "Moving Backward"
86            left_motor_speed = 100
87            right_motor_speed = -100
88        elif x_axis > -20:  # Right Turn
89            movement = "Turning Right"
90            left_motor_speed = -100
91            right_motor_speed = -100
92        elif x_axis < -20:  # Left Turn
93            movement = "Turning Left"
94            left_motor_speed = 100
95            right_motor_speed = 100
96
97        # Send PWM values to the motors
98        master.set_duty_cycle(MOTOR_LEFT_ID, left_motor_speed)
99        master.set_duty_cycle(MOTOR_RIGHT_ID, right_motor_speed)
100
101        # Print only if the state has changed
102        current_state = (movement, x_axis, y_axis, left_motor_speed, right_motor_speed)
103        if current_state != previous_state:
104            print(f"{movement} | X: {x_axis}, Y: {y_axis} | Left Motor: {left_motor_speed}, Right Motor: {right_motor_speed}")
105            previous_state = current_state  # Update the last printed state
106
107        # Stop motors if the joystick button is pressed
108        if button_pressed:
109            master.set_duty_cycle(MOTOR_LEFT_ID, 0)
110            master.set_duty_cycle(MOTOR_RIGHT_ID, 0)
111            print("Joystick button pressed, motors stopped.")
112            break
113
114        time.sleep(0.05)
115
116# Start Motor Control
117control_motors()

‍

Link to Acrome SMD Docs

For more details, visit: [Acrome SMD Documentation]

Conclusion

Teleoperation plays a critical role in modern robotics, enabling remote, real-time control of robotic systems in various industries. By integrating Acrome SMD motor controllers with mobile applications and keyboard-based teleoperation, users can achieve flexible and precise robotic motion control.

As robotics continues to advance, emerging technologies such as AI-powered automation, real-time video streaming, and cloud robotics will further enhance teleoperation capabilities. Acrome SMD-powered robots provide a reliable, scalable, and intelligent solution for industrial and research applications, paving the way for the next generation of remote-controlled robotic systems.

‍