Combine olives, shallot, celery, red peppers, capers, banana peppers (if using) and olive oil in a small bowl. Stir to combine. Season to taste with vinegar, salt, and pepper.
Tear out some of the doughy insides of the bread if it's too thick, and lay bread slices down face up on a cutting board. Spoon olive mixture on both the top and bottom halves. Layer meat and cheese onto the bottom half, then top with the top half. Cut into quarters and serve immediately, or for better flavor, wrap in plastic and allow bread to soak up juices for one hour before serving. Serve with chips and a pickle.