Nest sets and relativized closure properties

A nest set is the language generated by a context-free grammar whose rules are $\text{A → aB}\text{a}\text{?}\text{C}$ or A → ? (where a, ā are paired terminal symbols, and A, B, C are nonterminal symbols). They can be seen as the one-dimensional expressions of the recognizable sets of binary trees. We study their closure properties under relativized operations with respect to the Dyck set, which is the maximum set in the class. The operations considered are relativized versions of AFL operations, quotient, direct product, shuffle product, and some others. As an application we prove various closure properties of generalized parenthesis languages.