Microstructural compositions present in the weld zone of a structural steel were analyzed using nanoindentation and optical microscopy. Nanoindentation was performed on three different locations of the weld zone, including base metal (BM), heat-affected zone (HAZ) and weld metal (WM). Hundred points were indented on each location. The elastic modulus and hardness for each indenting point were then extracted from the nanoindentation results. Considering the mechanical properties as the characteristics of microstructures, statistical analysis with the deconvolution technique was applied to identify the microstructures and the corresponding volume fractions in BM, HAZ and WM. From the observations of optical microscopy and nanoindentation, it was shown that nanoindentation is acceptable in the verification of microstructural compositions and corresponding volume fractions in steel at three different locations of the weld zone. Moreover, the results showed that a microstructure of steel, ferrite, could be mechanically characterized in two types, low and high stiffness ferrite.