Some causes of facial swelling are:
obesity

allergic reaction (such as allergic rhinitis, hay fever, or a bee sting)

malnutrition (when severe)

sinusitis

drugs including prolonged use or allergic reaction to aspirin, antipyretics, penicillin, sulfa, glucocorticoids, or other drugs

blood transfusion reaction

angioedema from any cause including reaction to contrast media from radiologic tests

It sounds like an allergy of some sort. Try keeping a food diary to see if you can detect which food (if any) seem to trigger it. Be careful and pay attention if you start to get shortness of breath/throat becomes tight during one of these episodes as that could indicate a life threatenting reaction and you should seek medical care immediately.