Lightened Up Sausage Gravy

breakfast-style chicken sausage 1 Tbs. olive oil 1/2 cup mushrooms, diced 1 tsp. fresh thyme, chopped 1/2 tsp. fresh sage, chopped 2 Tbs. flour 2 cups milk (2% works best) 1/4 tsp. freshly grated nutmeg 1 Tbs. butter Salt & pepper to taste