We present the limit of a conventional photomask writer and new possibilities to meet various requirements of tight specifications for a sub-10 nm device. The issues of a variable shaped beam (VSB) writer and how to overcome the limit by computational techniques are discussed. Because VSB writing can use only one rectangular or triangular beam per shot, the complex design for computational lithography results in the increase of shot number to implement rounded or angled pattern. Based on model-based-fracturing, we have confirmed that the ideal curvilinear pattern can be optimized by using overlapping shots, and that they have the same patterning performance in mask and wafer. On the other hand, the multibeam writer can make ideally any kinds of shapes, even curvilinear design because the combination of small spots writes a design. In a real situation, each spot of multibeam writer is defined at fixed mesh and each beam has discrete dose level, so that there are fidelity errors if the pixel size is large or the dose level is not enough large. Here, we propose a “Buddhist cross” design as the evaluation pattern of digitization error in a multibeam writer. The “fidelity error” smaller than 0.5 nm error requires 5 nm pixel size and the required minimum number for dose level is 7 to implement a smaller error than 0.05 nm at one edge. To realize new technology for mass production, new data flow, model based pattern verification, and required computing power have been presented.