This papers derives finite sample results to assess the consistency of Generalized Pareto regression trees introduced by Farkas et al.  as tools to perform extreme value regression for heavy-tailed distributions. This procedure allows one to constitute clusters of similar tail behaviors depending on the value of the covariates, based on a recursive partition of the sample and simple model selection rules. The results we provide are obtained from concentration inequalities, and are valid for a finite sample size. A misspecification bias that arises from the use of a Peaks over Threshold approach is also taken into account. Moreover, the derived properties legitimate the pruning strategies, that is the model selection rules, used to select a proper tree that achieves a compromise between simplicity and good fit. The methodology is illustrated through a simulation study, and a real data application in insurance against natural disasters.