It depends. If your instructor has set a time limit for your quiz, when you click on the Begin quiz button, the clock begins to count down, and, therefore, you will NOT be able to exit the quiz. If the quiz is not timed, then you will be able to close the quiz window without any penalty.

By the way, although the time clock starts counting down immediately, the time at the top of the page is not updated until you submit a question. Be sure to pay attention to the time!