The aim of this study is to decrease a fishing boat's roll motion, which is adversely affected by disturbing hydrodynamic forces, by applying fin roll stabilizer. It is ensured that roll motion with nonlinear damping and restoring moment coefficients reaches the desired level by means of classical PID and modified PID algorithms. By Lyapunov's direct method, the key issue- stability which is needed during the course of sailing was examined considering initial conditions, and it was realized that the system was generally stable. Besides, NACA 0015 model was utilized for the fin roll stabilizer, and flow analysis was conducted by CFD method. According to the simulation results, when the same gains were practiced, modified PID controller algorithms were rather more efficient than conventional PID in the roll fin stabilizer system.