Hilbert space is a very powerfull mathematical tool that has proven to be incredibily useful in a wide range of applications. Matrix Hilebrt space is a new frame work that has been presented recently to perform a matrix inner product space when data observation is proposed as matrices and it is needed to analyse large dataset with complex and multi-dimensional structure. In this paper, a new norm associated to sequences in matrix Hilbert spaces is defined and the relation between this new norm and the norm obtained by matrix inner product is investigated. Also,it is discussed that although, some results and concepts in Hilbert spaces can be extended to the matrix Hilbert spaces, but counterexamples show that the results like Pythagorean theorem, Parallelogram law, Jordan-Von Neumann theorem are not hold in matrix Hilbert spaces. At last the reproducing kernel Hilbert space (RKHS) is extended to reproducing kernel matrix Hilbert space (RKMHS) and by considering some appropriate conditions on a matrix Hilbert space, an explicit structure of reproducing kernel is obtained and the RKMHS, H(K), which is obtained by reproducing kernel $K=K_{1}+K_{2}$ is characterized.