It depends upon the credit agreement you signed when you opened the account. Some credit card companies may waive all, a pro rata share, or none of your annual fee upon cancellation.