Lacking lexical information is a key barrier in developing high-performance Chinese Address Entity Recognition (AER) methods -- a critical foundation of many geospatial techniques. To this end, we propose a lexicon-based Chinese AER method using Collaborative Flat-Graph Transformer (CFGT). CFGT utilizes the lexical information to enhance the representation of character sequence of an Chinese address. First, it constructs two types of collaborative flat-graphs, flat-lattice and flat-shift, to capture the semantic information of self-matched lexical words (SMWs) and nearest contextual lexical words (NCWs) for characters. Then, their collaboration is achieved through a fusion layer. Secondly, it reinforce the effect of lexical enhancement using an improved relative position encoding, which complements the collaborative flat-graphs. Finally, it use Transformer to integrate Chinese character features with lexical information and CRF to derive final label predictions. Experimental results on various datasets show that the proposed CFGT method outperforms both the previous Chinese AER models and the state-of-the-art (SOTA) model in Chinese named entity recognition. The source code is available at https://github.com/hoppyNaut/CFGT.