In a 6-quart pot over medium heat, cook the bacon until crispy and browned, stirring occasionally.
Using a slotted spoon, transfer the bacon pieces to a plate and set aside.
To the pot, add the celery, shallots, cayenne, and ¼ teaspoon kosher salt. Cook for 7 to 10 minutes, or until the vegetables are almost tender, stirring occasionally.
Sprinkle the flour over the vegetables and cook for 1 minute, stirring constantly.
Slowly stir in the broth. Bring the soup to a simmer over high heat.
Add in the chicken and potatoes. Reduce the heat to medium and cook for 12 to 15 minutes, or until the chicken is cooked and the potatoes are tender, stirring occasionally. A thermometer inserted in the thickest piece of chicken should read 165°F.
Using tongs, transfer the chicken to a bowl and shred. Return the shredded chicken to the pot.
Stir in the milk, corn, and the remaining ½ teaspoon kosher salt. Cook for 3 minutes or until the corn is hot.
Remove from the heat and garnish the soup with basil and reserved bacon pieces. Serve with oyster crackers, if desired.