This paper presents a novel method to reconstruct continuous depth maps from light field data by solving a linear system. Conventional approaches treat depth map reconstruction as an optimization problem with discrete labels. The proposed method extracts raw depth information and confidence levels from the light field data using structure tensor. It also introduces a method to reduce the adverse effect of unreliable local estimations, helping to eliminate errors in specular areas and edges where depth values change abruptly. Experiments on synthetic and real light field data demonstrate the effectiveness of the proposed continuous depth map reconstruction method.