As it happens, one of the first crystal structure determinations was conducted to answer this question. Sometime before 1920, A. W. Hull of the GE Research Lab used x-ray diffraction to determine the crystal structures of the three elements that are ferromagnetic at room temperature: iron, cobalt, and nickel. The question was whether they all had the same crystal structure, which would suggest a direct connection between structure and magnetism. The answer was no: iron is body-centered cubic, cobalt is hexagonal close packed, nickel is face-centered cubic. Much later (1970s) it was found that non-crystalline materials (glassy metals, amorphous alloys) can also be ferromagnetic.
However, it is true that crystal structure affects magnetic properties of ferromagnetic elements, notably the magnetic anisotropy. Crystal structure also profoundly affects the magnetic structure and properties of antiferromagnetic and ferrimagnetic materials.