We present an efficient computational method to model fluid flow in the presence of random wall roughness. A random flow domain is represented by a stochastic indicator function having a smoothed profile perpendicular to roughness, and the random domain is discretized with a fixed non-conformal grid. This procedure introduces a stochastic force into the Navier-Stokes equations, and modifies the boundary conditions at the fluid-solid interface. We employ a high-order semi-implicit splitting scheme implemented in the context of a spectral/hp element method in order to discretize the physical domain. The stochastic roughness is treated as a second-order autoregressive process that is represented by a Karhunen-Loève expansion. A multi-element probabilistic collocation method is employed to solve the resulting stochastic Navier-Stokes equations. This method is applied to simulate external flow past a rough cylinder and internal Stokes flow between two parallel plates with random wall roughness. In the first problem, we develop an analytical solution for the asymptotic behavior of the lift coefficient CLCL to verify the results. In the second test-case, we compare the mean and the standard deviation of the velocity field to those obtained from a different method called stochastic mapping approach (SMA), developed by Tartakovsky and Xiu (2006).