If you still have a border on your form, then use the X. If not, then add a button to the form, double click, then enter Close()
One trick I use for deployed apps with no form border is to setup an event to handle the AdvancedHMI logo being double clicked.
Private Sub PictureBox1_DoubleClick(sender As Object, e As EventArgs) Handles PictureBox1.DoubleClick
Close()
End Sub