Indian Constitution, under Article 226 empowers High Courts of all states and union territories of the country to have power to issue directions, orders or writs (including habeas corpus, mandamus, quo warrantor, prohibition and certiorary), to any person, or authority or any government, within throughout the territory with regards to which the High Court has jurisdiction to exercise it's powers, for the purpose of the enforcement of any fundamental rights as conferred by the III Part of the Constitution and for any other purpose.
If the question arises with respect to the territorial jurisdiction of High Courts under Article 226, then it can be answered by looking into the 15th Amendment in Indian Constitution in the year 1963 which added clause (1-A) in Article 226, after clause (1) of the Article. According to clause (1-A), the power as given by clause (1) of Article 226, to issue direction, orders or writs to any person, authority or government, may also be exercisable by any High Court, within the territorial jurisdiction of which, the cause of action, for filing the application or petition for writ, direction or order under Article 226, has wholly or partly arisen. And for the exercise of this power by the High Court, within whose jursdiction cause of action has been arisen, it shall be immaterial of the fact whether the authority, or government against which, or the residence of the person against whom, the writ, or direction or order to be issued, falls within the jurisdiction of that High Court or not.
As far as the territorial jurisdiction of High Court under Article 226 is concerned, it is notable that as per Article 226(1), the power is to be exercised "throughout the territories in relation to which it exercises jurisdiction" that means if the person, authority or seat of the government and the cause of action also belonged to a state then only the High Court of that particular state would have jurisdiction over the matter and as per Article 226(2), the High Court within whose local limits of jurisdiction the cause of action arises (wholly or in part) that High Court would also have the jurisdiction even though the person, authority or government in relation to which the power is to be exercised, do not belong to that state but any other state. In such situation both High Courts would have jurisdiction over the matter.
The above adjudication is passed by the Supreme Court in Lt. Col. Khajoor Singh Vs Union of India, AIR 1961 SC 532.
In the present question, as the petitioner belong to Uttar Pradesh and the respondant (Government Department) belong to New Delhi, my answer to this situation would be that both High Courts, High Court of Delhi and that of Uttar Pradesh, would have jurisdiction over the matter, in view of Article 226(1) and there could be one more high court having jurisdiction for the same if the cause of action has arisen in another state, in view of Article 226(2).