sarproz-group

Sentinel TOPS threads

Tagged: 

Viewing 28 reply threads
  • Author
    Posts
    • #1133
      Yuxiao
      Spectator

        Dear Matus and Ivana,

        Hi Matus, please update to the newest sarproz version and the question of black borders should go away. You shouldn’t need to change any oversampling factors. Please let me know if the new one works fine for you.

        Hi Ivana, please also try to update to the newest sarproz for your future processing. For next steps of your processing, may I ask you to take a look at the messages in Matlab, especially when you click the button ‘coregistration’? If any errors happened in the future while doing TOPS post processing, you should stop and let me know the error. It will be meaningless to carry on and generate interferograms if any error occurred in TOPS post processing.

        About the black strips in your slave coregistered image, I will need a screenshot of your module SLC data import where I need to see the center of your scene and the length/width of the scene you selected. I need to exactly replicate your area in order to investigate the error going on there. Please let me know.

        Thank you all for your testings,

        Yours,
        Yuxiao

      • #1151
        matusbakon
        Participant

          Hi Yuxiao,

          black borders has gone, thank you, however, footprints are now slightly different and they are not covering whole AOI. I’ll try to search for adjacent track based on Daniele’s suggestion from other forum thread and I’ll continue on testing.

          Cheers

          Matus

        • #1187
          matusbakon
          Participant

            hi there, apparently there’s a problem with renaming the Sentinel-1 images with the same dates.

            when the duplicates are deleted from SLC directory, it is possible to continue with proccesing.

            lasterror.mat and log files attached

            cheers

            Attachments:
            You must be logged in to view attached files.
          • #1189
            periz
            Keymaster

              We are working at stitching subsequent images together. Up to now Sarproz never did it, we always worked with single footprints.
              The reason for renaming images acquired on the same day was a quick and dirty solution for managing TanDEM-x bistatic pairs without changing the naming convention.
              What you can do at this moment is working on them separately and stitch them together later…

            • #1190
              matusbakon
              Participant

                finally, i get some promisingly looking set of images and interferograms, but also:

                1) corregistration refinement error (applied experimentally),
                2) interferograms plotted through Small Area Processing differs when “View SLC” and “Read and Plot Interfs” functions are used,
                and can be seen in attached zip (second folder with time-ordered print screens)

                thanks!

                small hint: selecting subswath number depends on ascending/descending geometry (the order of numbers 1, 2, 3), while choosing subswath from map-overlay.kml (or quick-look.png) that are attached to every image, one would expect that the order of numbers 1, 2, 3 would be always from left-to-right (only in the case of ascending acquisition) not right-to-left (descending)

                • #1194
                  Yuxiao
                  Spectator

                    Hi Matus,

                    I am still looking into your results and logs, but I reply here what I’ve finished looking and I will keep on investigating and reply you the rest.

                    1. Co-registration-error: Sorry, this error was due to a bug in the code that deleted the SLC file so that the file for coregistration refinement is missing. I will kill this bug for the next sarproz release in the coming days.

                    2. (by looking at the images you’ve send me), one is the PHASE of a single SLC image, the other one is the interferogram. They are totally two different things.

                    About the subswath number: we will merge them in the future…

                    yuxiao

                • #1191
                  matusbakon
                  Participant

                    sorry, the file was too big to be included in attachement

                    https://www.dropbox.com/s/m30bbbhcoarrreh/tests.zip?dl=0

                  • #1192
                    ivca-cz
                    Participant

                      Hello,

                      sorry for my long pause, I was out.

                      Now I am trying the processing again, with the compiled version of Aug 31st (as the update function does not work for me in the matlab version).

                      I decided to delete those problematic images, as they were problematic due to only partial overlap with the others in the AOI. The process of selecting the AOI is a bit unstable as sometimes the number of lines/pixels changes and sometimes it does not (after the change of the radius of the AOI), sometimes I get negative number of lines etc. The problems were (at least I think) caused by the wrong computation of overlapping area, causing that the images were smaller (in the azimuth direction) than the other ones.

                      My suggestion here would be not to limit the crop to the overlap of all images, but to drop those images which do not contain the whole AOI (or to make the user choose), but this would probably be too complex and practically half of the problem will (the azimuth) be solved after making the images to stitch together.

                      So, the black lines were probably caused by having two images in the set that did not cover the whole AOI.

                      Still, the update mode does not work so every attempt to reprocess it takes practically half a day (do you need logfile for this?)

                      I also have problems with geocoding of all existent interferograms together (“please select a parameter first”), but not sure if this problem is related to S1/TOPS or if it is not related only to my computer (last time (2 weeks ago) I had to abrupt this process by killing whole sarproz).

                      When processing itfs (not tried PS yet) I get some phase jumps, probably related to one image (using MST, I can see them in two itfs with a common image). Uploading them in an attachment. And in some other itfs, a part of the itf has significantly lower coherence, which can also be caused by a coregistration error. Most of the itfs are ok.

                      Is there any possibility to find (spatially) the start and the end of a burst, also in the interferograms which do not have these problems?

                      Is there anything I can do to help solve these problems?

                      Thank you, Ivana

                      • #1196
                        Yuxiao
                        Spectator

                          Hi Ivana,

                          Recent days I am working on killing the bugs that are possibly occurred because of partial overlap images. In the next few days a new version should be released and it should work better with those partial overlapped images.

                          Can you provide the log file of geocoding all interferograms and what the error is? So we can look into detail on that…

                          About the phase jump in your interferogram, this is what I will look into in the next few days. These are very likely due to the bugs in the algorithms that I will need to improve.
                          The whole idea of TOPS is to stitch burst flawlessly.
                          For me able to solve the problem in your interferogram, I will need your help on this:

                          Provide me a list of image dates that you find (even a slightly) errors/problems in interferograms;
                          and I need a screenshot of when you’re doing SLC data import, I need to see the center of your scene and the length/width of the scene you selected. I need to exactly replicate your area in order to investigate the error in interferogram.

                          Thank you Ivana!

                          yuxiao

                      • #1193
                        ivca-cz
                        Participant

                          sorry, the file was also too large 🙂

                          http://www.insar.cz/itfs.zip

                          Ivana

                        • #1197
                          ivca-cz
                          Participant

                            Hello Yuxiao,

                            OK, trying to attach the logfile (long).

                            the image with the phase jump is 20141223.

                            and the images with wrong coherence (at the North) are 20150820-20150901, 20150703-20150715, 20150808-20150901, 20150727-20150808, 20150715-20150727.

                            regarding the area: the center is now (if I remember well, is it possible to get the values without reprocessing it all?) 50.54, 13.5 and the size is probably 18 km.

                            Ivana

                          • #1198
                            ivca-cz
                            Participant

                              once again the attachment

                              Attachments:
                              You must be logged in to view attached files.
                            • #1203
                              ivca-cz
                              Participant

                                Hello Yuxiao,

                                just for sure, I am reporting failed coregistration (for 1 image) on different scene set. I cannot find any reason (error) for this. And the error does not let me go further.

                                Thank you, Ivana

                                • #1205
                                  Yuxiao
                                  Spectator

                                    Hi Ivana,

                                    I can’t see your log in the forum… Can you find another way to pass the log file?

                                    thanks,

                                    yuxiao

                                  • #1206
                                    Yuxiao
                                    Spectator

                                      Hi Ivana,

                                      by looking at the log files, I would suggest the following:

                                      Checking in /EXT folder the file 20150123_20150324_init.off and check if there is anything written inside.

                                      It would also be helpful to check the intensity of both the master image and slave image. the master will be in FITTED folder as the name 20150213.jpg. The slave image will be in EXT folder by the name of 20150324_VV.slc.

                                      These info could be helpful to understand why the coregistration failed for this particular image.

                                      By the way, I see that coregistration and tops-post-processing is finished with the rest of the images so you should be able to carry on with the rest of the processing without the failed coregistration image 20150324 at this moment.

                                  • #1207
                                    ivca-cz
                                    Participant

                                      Yes, the .off file is present and looks reasonably.

                                      And slave image is also in the EXT folder, except for the fact that it is not VV, but HH, and the corresponding jpg looks reasonably, similar to the master.

                                      By the way (the processing is not important, I am doing it just to check and see), I could not open the site processing (did you see it from the logfile?) – or is there any other possibility how to continue working without an image? Manually adjusting DataSet.txt?

                                      Ivana

                                    • #1220
                                      ivca-cz
                                      Participant

                                        Hello again,

                                        using the compiled version of Aug 31 (no newer), I was processing S1 data, and the interferograms and everything were nice using automatically-downloaded SRTM. Now, I used my own DEM, and the interferograms look weird (see the itf and the DEM visualization).

                                        I admit I have a DEM with such a weird form (corner cut) and I used it, but at the same time I realized that this DEM is not the right one and gave the software (at least three times to be sure) the new one, which has normal rectangular shape, but since that time, I get this form of itfs. I re-made the interferograms (and all the steps with green button) also at least three times, and meanwhile, also restarted sarproz, so I think that the cause should not be in the wrong DEM used for a while.

                                        I understand that the DEM may not cover the whole itf (although I think it does). But this shape is too weird.

                                        Or is that the “not-ready-topography for TOPS”, as mentioned earlier?

                                        Thank you, Ivana

                                        Attachments:
                                        You must be logged in to view attached files.
                                      • #1222
                                        ivca-cz
                                        Participant

                                          I am sorry, that was not the compiled version, but older matlab version, so that is probably the problem… trying it once more in the newer version.

                                          I.

                                        • #1223
                                          ivca-cz
                                          Participant

                                            No, I am getting exactly the same results also in the newest compiled version… together with lots of

                                            stima_freq: WARNING!!

                                            NaNs detected in input matrix: forcing them to zero!!

                                            I do know where the problem can be…

                                          • #1225
                                            periz
                                            Keymaster

                                              Ivana,
                                              first of all, a general comment: you have all needed tools to investigate your problem…!!
                                              You can plot the DEM in geographic coordinates, and you know you should expect 4 black crosses that give you an idea about where is the footprint of your data w.r.t. the DEM. I do not see them in the picture above, and this should be a first sign that probably the DEM is smaller that the footprint.
                                              In any case, you can proceed and process the DEM in SAR coordinates. Then you can plot it and there you will have the confirm of what is going on. In fact, I expect that the weird pattern you see in your interferogram will also be visible in the DEM in SAR coordinates.
                                              Then the sw now tells you if any NaN data is detected in the interferogram. This could also happen in case one of the images was containing NaNs data at the border (in case of discrepancy between orbital data and actual image registration on the Master). You can verify the reason e.g. by adding removing InSAR processing options (e.g. DEM removal or not).
                                              Sorry for the many NaNs messages, we fixed it.

                                              To conclude: Sarproz at this moment is managing external DEMs in a kind of basic and blind way. That is, the ext DEM is taken as it is inputed and it is resampled in SAR coordinates. If the DEM is smaller than the footprint, you get NaNs data. In other words, you are expected to use an external DEM which is containing the footprint. Question: should we include an automatic merging between ext. DEM and SRTM where the first is missing/not defined?

                                              About TOPS and DEM at this moment our suspects were not confirmed by any experiments. In other words, it looks ok so far.

                                            • #1228
                                              mspyhm2001
                                              Participant

                                                Dear Yuxiao and other my friends, hi
                                                ————————————————–
                                                1-
                                                I am working with Sentinel-1 TOPS data processing on SARPROZ hardly. 🙂
                                                I`ve tested 3 case on various regions. I am facing to a common problem in all cases.
                                                In each sub-scene, the half of scene be null automatically (see attached kmz file).
                                                My Sarproz installed version is 19-Aug-2015
                                                If i need to update my SARPROZ, pls help me.
                                                ————————————————–
                                                2-
                                                I have used 6*1 Rg*Az for multilooking on TOPS data. Is it correct?

                                                Attachments:
                                                You must be logged in to view attached files.
                                                • #1233
                                                  Yuxiao
                                                  Spectator

                                                    Dear mspyhm2001.

                                                    First of all, you might noticed that you need to zip/tar your file before upload them, or you can use an external link.

                                                    You can update sarproz with ‘version management’ inside ‘data selection’.

                                                    The multilook can be selected as whatever you want that fits your need.

                                                    I attach here a link of me extracting more or less a same area of yours using your data and your date with the updated version of sarproz.

                                                    yuxiao

                                                  • #1235
                                                    mspyhm2001
                                                    Participant

                                                      I use compiled version and i trying with “version management”, but it not work for update

                                                  • #1236
                                                    periz
                                                    Keymaster

                                                      If you want to get it working, please read the messages given by the sw.
                                                      The sw always tells you what’s going on and whether anything is wrong.
                                                      In case of unexpected errors, all error details are saved in a .mat file.
                                                      If you want to get assistance, zip .mat and .log files and attach them here.
                                                      If no errors occurred, just zip and attach the .log file here.

                                                      • #1237
                                                        mspyhm2001
                                                        Participant

                                                          It looks like none of the servers is accessible for me (attached zip file)

                                                          Attachments:
                                                          You must be logged in to view attached files.
                                                      • #1239
                                                        ivca-cz
                                                        Participant

                                                          Hello,

                                                          I have also experienced interferograms with zero part of the area (in azimuth). I think that it can be (in some cases) solved by cropping larger area (in the area selection) with the radius over cca 12 km. With the radius of 5 km, I experienced this problems (in two datasets). There is no error here except for the “azimuth stretch” warning. All the cropped and coregistered slaves look reasonably in this case.

                                                          In one dataset, I have the problem of selecting larger area because in that case, I get into partial overlap and I could not deal with that because I do not get any info about in which direction the partial overlap is a problem and the process of area selection is messy (could you please enter into the logfile something like “selection ok” so that we recognize the last (good) selection from the previous (bad) ones, telling about the out-of-area scenes?)

                                                          Corresponds to the version of Sep-7, as in Sep-12 version I have another problem that does not allow me to process the interferograms.

                                                          Regards, Ivana

                                                          • #1243
                                                            Yuxiao
                                                            Spectator

                                                              Dear Ivana,

                                                              Can you be more explicit on your problems, like, uploading the results and images that are problematic? In this way we can better debug.

                                                              The sw cannot yet merge adjacent strips, we will work on that in the future.

                                                              yuxiao

                                                          • #1244
                                                            ivca-cz
                                                            Participant

                                                              Hi Yuxiao,

                                                              I am attaching one image from the FITTED directory (they all look approximately the same) and one image from the INTERFEROGRAMS directory (they do not look the same as expected, but the half-zero pattern is the same in all the interferograms), and the logfiles (hope they are the right ones). Please let me know what else you need.

                                                              Ivana

                                                            • #1245
                                                              ivca-cz
                                                              Participant
                                                              • #1303
                                                                mspyhm2001
                                                                Participant

                                                                  Dear Colleagues, I work on S1-TOPS mode. Please see attached zip file.
                                                                  I cant process all my area in any region. i am using compiled version 12-sep-2015 and trying for update to 27-oct-2015 but i cant connect to server. please help me

                                                                • #1305
                                                                  mspyhm2001
                                                                  Participant
                                                                  • #1320
                                                                    periz
                                                                    Keymaster

                                                                      Hi,

                                                                      we experienced some problems related to the forum (missing replies, delays in notifications).
                                                                      Please try again to download the last sw version. If you fail it, write directly to the admin and you will receive the update

                                                                      dp

                                                                    • #1368
                                                                      ivca-cz
                                                                      Participant

                                                                        Hello,

                                                                        I am attaching an error I encountered more times (but sometimes not; in different areas), I am processing an area close to image border; what is the problem here and how shall I avoid it?

                                                                        If I understand it well, it concerns the number of bursts…?

                                                                        Thank you, Ivana

                                                                        Attachments:
                                                                        You must be logged in to view attached files.
                                                                        • #1370
                                                                          Yuxiao
                                                                          Spectator

                                                                            Hi Dear Ivana,

                                                                            Thanks for the testing, this is indeed a bug and I’ve fixed it.

                                                                            I’ll submit the fixed code for sarproz.

                                                                            Between now and the publishing of new version, as a temporarily work around, this bug can generally be avoided by not selecting areas close to the border of Sentinel image.

                                                                            Also in your log I see that you have sentinel images on the same data in adjacent areas. We are also developing code for merging adjacent tops images at this moment.

                                                                            Thank again for the feedback,

                                                                            Yuxiao

                                                                        • #3446
                                                                          jeenujohn
                                                                          Participant

                                                                            hi,
                                                                            my master image is not coorectly extracting .Slave images also not extracted .the error showing is AOI line is not extracted

                                                                          • #3592
                                                                            dedikryan
                                                                            Participant

                                                                              hi,
                                                                              im newer in using sarproz. while im fill in SLC DATA processing, i need to change the subswath. but the update is not possible. how can i change that subswath? i use sentinel 1 data.
                                                                              thank you.

                                                                              regards,
                                                                              Made

                                                                              Attachments:
                                                                              You must be logged in to view attached files.
                                                                            • #3594
                                                                              periz
                                                                              Keymaster

                                                                                You are in update mode: a Master is present and you chose to keep it. Since that Master was extracted in a given subswath, you cannot change it.
                                                                                If you want to change the subswath, you have to remove the previous Master.
                                                                                How to do it?
                                                                                Check the “All” flag and then press “Set orbits”. The sw will warn you that it is going to delete the existing files/settings.
                                                                                Alternatively, you can do it manually: before importing new SLC/subswath/whatever, put yourself in the initial conditions: remove all files but the SLC folder with the original SLC images
                                                                                best

                                                                              • #3607
                                                                                Arash
                                                                                Participant

                                                                                  Hello dears
                                                                                  My study area is located in 2 subswaths of Sentinel images(iw1 & iw2). I am the new user of Sarproz and I do not know how to do this. If the steps or tutorial are available, please help me.
                                                                                  Thanks.

                                                                              Viewing 28 reply threads
                                                                              • You must be logged in to reply to this topic.