@sophiamyang Regarding the Mistral 8x 7b model, does each expert have 7 billion parameters, or is the total sum of parameters across all 8 experts 7 billion?