Weapons are more than common where I'm from. But magic is considered a secret, unknown to most of the world. Among the society that uses it, making primary use of weapons is considered foul play and cheating. Magi are supposed to fight with magic. Even training in martial arts is looked down upon.

Either way, magic is powerful enough to overwhelm someone who does not use magic. For example, I'm one of the few who can use a Reality Marble. If I can drag people into a world that exists entirely under my control, what defense do people have against that?