The problem of the Rayleigh wave scattering on the three- and two-dimensional statistical roughness of isotropic solid is solved in the Born approximation of the perturbation theory in a roughness amplitude. Statistically homogeneous and isotropic roughness is described by a correlation function which has a form of the exponentially modulated Chebyshev-Laguerre polynomials sum. This approximation of a correlator is new, but it agrees with experimental data well. A new form of the Rayleigh wave diffraction is theoretically obtained. It is characterized by a violation of the Rayleigh law of scattering and of ordinary laws of the resonance and diffuse scatterings. The fundamental physical conception that a wave does not sense the structure of an irregularity in a long-wavelength scattering, when the wavelength is much greater than the character size of the irregularity, that is in the Rayleigh scattering, is violated as well. From the physical point of view this new diffraction is astrong modulation of the Rayleigh, resonance and diffuse scatterings by a roughness form.