Abstract: Aiming at the path planning problem of collision avoidance and overtaking for intelligent vehicles, this paper proposes an innovative artificial potential field method based on visual perception of the road environment by using monocular cameras so as to provide necessary information for path planning. For structured roads, inverse perspective transformation is adopted to eliminate the projection characteristics of visual images. Thus, the parameters of lanes are determined according to the parallel relationship of lane lines and the algorithm of random sampling consensus, which improves the accuracy of lane detection. Based on this, a ground ranging model is proposed and used to measure the width of a lane. Next, a YOLO-3DBOX network structure is proposed to obtain the information of vehicles ahead on the road. At first, YOLOv4 is adopted to achieve two-dimensional detection of the vehicle. Then, with the help of the idea of Deep3DBOX, prediction bias and geometric projection constraints are used to regress the dimension, orientation angle and center coordinates of the vehicle, and finally to achieve three-dimensional detection of the target. So far, the obtained complete perception information of roads and vehicles can be used to design artificial potential fields for autonomous driving. Firstly, the gravitational potential field of the main lane is designed to guide the controlled vehicle to drive forward in the main lane. Then, according to the driving specifications for vehicles on road, the road boundary in the potential field is designed to avoid the controlled vehicle crossing the center line of the lane or the road boundary. Next, the collision avoidance area is designed according to the relative velocity between the controlled vehicle and the vehicle in front to constrain the range of the potential field of the road boundary and the repulsion field of obstacles. Meanwhile, the repulsion field of obstacles is constructed by using exponential and trigonometric functions. According to the kinematic constraints of the vehicle, the sine function is used to adjust the transverse and longitudinal parameters to ensure that the planned path can avoid obstacles smoothly and conform to the steering characteristics of the vehicle. In order to verify the effectiveness of the proposed algorithm, the planned paths are compared and analyzed under various simulation conditions. Simulation results show that the improved artificial potential field algorithm fully considers the road boundary and vehicle constraints, and can complete path planning on straight and curved roads. The controlled vehicle can avoid and overtake static and dynamic obstacles at different speeds. In the process of obstacle avoidance, the controlled vehicle always keeps a sufficient safe distance from the obstacle, and returns to the original lane in time after overtaking. In addition, Carsim/Simulink software is used to carry out co-simulation of path tracking. The maximum of both lateral error and heading angle deviation of the vehicle are 0.094 m and 0.021 rad in the tracking process, which proves that the planned path can guarantee that the vehicle has good tracking performance. Finally, in order to further verify the feasibility of the system, an experiment is carried out on the FR-08 wire chassis platform. The experimental results show that the proposed algorithm can plan and track the path for obstacle avoidance according to information of the road and the vehicle ahead. The maximum lateral error between the actual trajectory and the planned path is 0.105 m, and the maximum error of the heading angle is 0.092 rad. The simulation and experiment results show that the proposed algorithm for collision avoidance and overtaking of intelligent vehicles can realize the information perception of road environment, and, at the same time, plan and track a reasonable path that conforms to the kinematic characteristics of vehicles.