sarproz-group

Sentinel TOPS threads

Tagged: 

This topic contains 33 replies, has 5 voices, and was last updated by  Yuxiao 1 year, 8 months ago.

  • Author
    Posts
  • #1133

    Yuxiao
    Moderator

    Dear Matus and Ivana,

    Hi Matus, please update to the newest sarproz version and the question of black borders should go away. You shouldn’t need to change any oversampling factors. Please let me know if the new one works fine for you.

    Hi Ivana, please also try to update to the newest sarproz for your future processing. For next steps of your processing, may I ask you to take a look at the messages in Matlab, especially when you click the button ‘coregistration’? If any errors happened in the future while doing TOPS post processing, you should stop and let me know the error. It will be meaningless to carry on and generate interferograms if any error occurred in TOPS post processing.

    About the black strips in your slave coregistered image, I will need a screenshot of your module SLC data import where I need to see the center of your scene and the length/width of the scene you selected. I need to exactly replicate your area in order to investigate the error going on there. Please let me know.

    Thank you all for your testings,

    Yours,
    Yuxiao

  • #1151

    matusbakon
    Participant

    Hi Yuxiao,

    black borders has gone, thank you, however, footprints are now slightly different and they are not covering whole AOI. I’ll try to search for adjacent track based on Daniele’s suggestion from other forum thread and I’ll continue on testing.

    Cheers

    Matus

  • #1187

    matusbakon
    Participant

    hi there, apparently there’s a problem with renaming the Sentinel-1 images with the same dates.

    when the duplicates are deleted from SLC directory, it is possible to continue with proccesing.

    lasterror.mat and log files attached

    cheers

    Attachments:
    You must be logged in to view attached files.
  • #1189

    periz
    Keymaster

    We are working at stitching subsequent images together. Up to now Sarproz never did it, we always worked with single footprints.
    The reason for renaming images acquired on the same day was a quick and dirty solution for managing TanDEM-x bistatic pairs without changing the naming convention.
    What you can do at this moment is working on them separately and stitch them together later…

  • #1190

    matusbakon
    Participant

    finally, i get some promisingly looking set of images and interferograms, but also:

    1) corregistration refinement error (applied experimentally),
    2) interferograms plotted through Small Area Processing differs when “View SLC” and “Read and Plot Interfs” functions are used,
    and can be seen in attached zip (second folder with time-ordered print screens)

    thanks!

    small hint: selecting subswath number depends on ascending/descending geometry (the order of numbers 1, 2, 3), while choosing subswath from map-overlay.kml (or quick-look.png) that are attached to every image, one would expect that the order of numbers 1, 2, 3 would be always from left-to-right (only in the case of ascending acquisition) not right-to-left (descending)

    • #1194

      Yuxiao
      Moderator

      Hi Matus,

      I am still looking into your results and logs, but I reply here what I’ve finished looking and I will keep on investigating and reply you the rest.

      1. Co-registration-error: Sorry, this error was due to a bug in the code that deleted the SLC file so that the file for coregistration refinement is missing. I will kill this bug for the next sarproz release in the coming days.

      2. (by looking at the images you’ve send me), one is the PHASE of a single SLC image, the other one is the interferogram. They are totally two different things.

      About the subswath number: we will merge them in the future…

      yuxiao

      • This reply was modified 1 year, 11 months ago by  Yuxiao.
      • This reply was modified 1 year, 11 months ago by  periz.
  • #1191

    matusbakon
    Participant

    sorry, the file was too big to be included in attachement

    https://www.dropbox.com/s/m30bbbhcoarrreh/tests.zip?dl=0

  • #1192

    ivca-cz
    Participant

    Hello,

    sorry for my long pause, I was out.

    Now I am trying the processing again, with the compiled version of Aug 31st (as the update function does not work for me in the matlab version).

    I decided to delete those problematic images, as they were problematic due to only partial overlap with the others in the AOI. The process of selecting the AOI is a bit unstable as sometimes the number of lines/pixels changes and sometimes it does not (after the change of the radius of the AOI), sometimes I get negative number of lines etc. The problems were (at least I think) caused by the wrong computation of overlapping area, causing that the images were smaller (in the azimuth direction) than the other ones.

    My suggestion here would be not to limit the crop to the overlap of all images, but to drop those images which do not contain the whole AOI (or to make the user choose), but this would probably be too complex and practically half of the problem will (the azimuth) be solved after making the images to stitch together.

    So, the black lines were probably caused by having two images in the set that did not cover the whole AOI.

    Still, the update mode does not work so every attempt to reprocess it takes practically half a day (do you need logfile for this?)

    I also have problems with geocoding of all existent interferograms together (“please select a parameter first”), but not sure if this problem is related to S1/TOPS or if it is not related only to my computer (last time (2 weeks ago) I had to abrupt this process by killing whole sarproz).

    When processing itfs (not tried PS yet) I get some phase jumps, probably related to one image (using MST, I can see them in two itfs with a common image). Uploading them in an attachment. And in some other itfs, a part of the itf has significantly lower coherence, which can also be caused by a coregistration error. Most of the itfs are ok.

    Is there any possibility to find (spatially) the start and the end of a burst, also in the interferograms which do not have these problems?

    Is there anything I can do to help solve these problems?

    Thank you, Ivana

    • #1196

      Yuxiao
      Moderator

      Hi Ivana,

      Recent days I am working on killing the bugs that are possibly occurred because of partial overlap images. In the next few days a new version should be released and it should work better with those partial overlapped images.

      Can you provide the log file of geocoding all interferograms and what the error is? So we can look into detail on that…

      About the phase jump in your interferogram, this is what I will look into in the next few days. These are very likely due to the bugs in the algorithms that I will need to improve.
      The whole idea of TOPS is to stitch burst flawlessly.
      For me able to solve the problem in your interferogram, I will need your help on this:

      Provide me a list of image dates that you find (even a slightly) errors/problems in interferograms;
      and I need a screenshot of when you’re doing SLC data import, I need to see the center of your scene and the length/width of the scene you selected. I need to exactly replicate your area in order to investigate the error in interferogram.

      Thank you Ivana!

      yuxiao

  • #1193

    ivca-cz
    Participant

    sorry, the file was also too large 🙂

    http://www.insar.cz/itfs.zip

    Ivana

  • #1197

    ivca-cz
    Participant

    Hello Yuxiao,

    OK, trying to attach the logfile (long).

    the image with the phase jump is 20141223.

    and the images with wrong coherence (at the North) are 20150820-20150901, 20150703-20150715, 20150808-20150901, 20150727-20150808, 20150715-20150727.

    regarding the area: the center is now (if I remember well, is it possible to get the values without reprocessing it all?) 50.54, 13.5 and the size is probably 18 km.

    Ivana

  • #1198

    ivca-cz
    Participant

    once again the attachment

    Attachments:
    You must be logged in to view attached files.
  • #1203

    ivca-cz
    Participant

    Hello Yuxiao,

    just for sure, I am reporting failed coregistration (for 1 image) on different scene set. I cannot find any reason (error) for this. And the error does not let me go further.

    Thank you, Ivana

    • #1205

      Yuxiao
      Moderator

      Hi Ivana,

      I can’t see your log in the forum… Can you find another way to pass the log file?

      thanks,

      yuxiao

    • #1206

      Yuxiao
      Moderator

      Hi Ivana,

      by looking at the log files, I would suggest the following:

      Checking in /EXT folder the file 20150123_20150324_init.off and check if there is anything written inside.

      It would also be helpful to check the intensity of both the master image and slave image. the master will be in FITTED folder as the name 20150213.jpg. The slave image will be in EXT folder by the name of 20150324_VV.slc.

      These info could be helpful to understand why the coregistration failed for this particular image.

      By the way, I see that coregistration and tops-post-processing is finished with the rest of the images so you should be able to carry on with the rest of the processing without the failed coregistration image 20150324 at this moment.

  • #1207

    ivca-cz
    Participant

    Yes, the .off file is present and looks reasonably.

    And slave image is also in the EXT folder, except for the fact that it is not VV, but HH, and the corresponding jpg looks reasonably, similar to the master.

    By the way (the processing is not important, I am doing it just to check and see), I could not open the site processing (did you see it from the logfile?) – or is there any other possibility how to continue working without an image? Manually adjusting DataSet.txt?

    Ivana

  • #1220

    ivca-cz
    Participant

    Hello again,

    using the compiled version of Aug 31 (no newer), I was processing S1 data, and the interferograms and everything were nice using automatically-downloaded SRTM. Now, I used my own DEM, and the interferograms look weird (see the itf and the DEM visualization).

    I admit I have a DEM with such a weird form (corner cut) and I used it, but at the same time I realized that this DEM is not the right one and gave the software (at least three times to be sure) the new one, which has normal rectangular shape, but since that time, I get this form of itfs. I re-made the interferograms (and all the steps with green button) also at least three times, and meanwhile, also restarted sarproz, so I think that the cause should not be in the wrong DEM used for a while.

    I understand that the DEM may not cover the whole itf (although I think it does). But this shape is too weird.

    Or is that the “not-ready-topography for TOPS”, as mentioned earlier?

    Thank you, Ivana

    Attachments:
    You must be logged in to view attached files.
  • #1222

    ivca-cz
    Participant

    I am sorry, that was not the compiled version, but older matlab version, so that is probably the problem… trying it once more in the newer version.

    I.

  • #1223

    ivca-cz
    Participant

    No, I am getting exactly the same results also in the newest compiled version… together with lots of

    stima_freq: WARNING!!

    NaNs detected in input matrix: forcing them to zero!!

    I do know where the problem can be…

  • #1225

    periz
    Keymaster

    Ivana,
    first of all, a general comment: you have all needed tools to investigate your problem…!!
    You can plot the DEM in geographic coordinates, and you know you should expect 4 black crosses that give you an idea about where is the footprint of your data w.r.t. the DEM. I do not see them in the picture above, and this should be a first sign that probably the DEM is smaller that the footprint.
    In any case, you can proceed and process the DEM in SAR coordinates. Then you can plot it and there you will have the confirm of what is going on. In fact, I expect that the weird pattern you see in your interferogram will also be visible in the DEM in SAR coordinates.
    Then the sw now tells you if any NaN data is detected in the interferogram. This could also happen in case one of the images was containing NaNs data at the border (in case of discrepancy between orbital data and actual image registration on the Master). You can verify the reason e.g. by adding removing InSAR processing options (e.g. DEM removal or not).
    Sorry for the many NaNs messages, we fixed it.

    To conclude: Sarproz at this moment is managing external DEMs in a kind of basic and blind way. That is, the ext DEM is taken as it is inputed and it is resampled in SAR coordinates. If the DEM is smaller than the footprint, you get NaNs data. In other words, you are expected to use an external DEM which is containing the footprint. Question: should we include an automatic merging between ext. DEM and SRTM where the first is missing/not defined?

    About TOPS and DEM at this moment our suspects were not confirmed by any experiments. In other words, it looks ok so far.

  • #1228

    mspyhm2001
    Participant

    Dear Yuxiao and other my friends, hi
    ————————————————–
    1-
    I am working with Sentinel-1 TOPS data processing on SARPROZ hardly. 🙂
    I`ve tested 3 case on various regions. I am facing to a common problem in all cases.
    In each sub-scene, the half of scene be null automatically (see attached kmz file).
    My Sarproz installed version is 19-Aug-2015
    If i need to update my SARPROZ, pls help me.
    ————————————————–
    2-
    I have used 6*1 Rg*Az for multilooking on TOPS data. Is it correct?

    • This reply was modified 1 year, 11 months ago by  mspyhm2001.
    Attachments:
    You must be logged in to view attached files.
    • #1233

      Yuxiao
      Moderator

      Dear mspyhm2001.

      First of all, you might noticed that you need to zip/tar your file before upload them, or you can use an external link.

      You can update sarproz with ‘version management’ inside ‘data selection’.

      The multilook can be selected as whatever you want that fits your need.

      I attach here a link of me extracting more or less a same area of yours using your data and your date with the updated version of sarproz.

      yuxiao

      • This reply was modified 1 year, 11 months ago by  Yuxiao.
    • #1235

      mspyhm2001
      Participant

      I use compiled version and i trying with “version management”, but it not work for update

  • #1236

    periz
    Keymaster

    If you want to get it working, please read the messages given by the sw.
    The sw always tells you what’s going on and whether anything is wrong.
    In case of unexpected errors, all error details are saved in a .mat file.
    If you want to get assistance, zip .mat and .log files and attach them here.
    If no errors occurred, just zip and attach the .log file here.

    • #1237

      mspyhm2001
      Participant

      It looks like none of the servers is accessible for me (attached zip file)

      Attachments:
      You must be logged in to view attached files.
  • #1239

    ivca-cz
    Participant

    Hello,

    I have also experienced interferograms with zero part of the area (in azimuth). I think that it can be (in some cases) solved by cropping larger area (in the area selection) with the radius over cca 12 km. With the radius of 5 km, I experienced this problems (in two datasets). There is no error here except for the “azimuth stretch” warning. All the cropped and coregistered slaves look reasonably in this case.

    In one dataset, I have the problem of selecting larger area because in that case, I get into partial overlap and I could not deal with that because I do not get any info about in which direction the partial overlap is a problem and the process of area selection is messy (could you please enter into the logfile something like “selection ok” so that we recognize the last (good) selection from the previous (bad) ones, telling about the out-of-area scenes?)

    Corresponds to the version of Sep-7, as in Sep-12 version I have another problem that does not allow me to process the interferograms.

    Regards, Ivana

    • #1243

      Yuxiao
      Moderator

      Dear Ivana,

      Can you be more explicit on your problems, like, uploading the results and images that are problematic? In this way we can better debug.

      The sw cannot yet merge adjacent strips, we will work on that in the future.

      yuxiao

  • #1244

    ivca-cz
    Participant

    Hi Yuxiao,

    I am attaching one image from the FITTED directory (they all look approximately the same) and one image from the INTERFEROGRAMS directory (they do not look the same as expected, but the half-zero pattern is the same in all the interferograms), and the logfiles (hope they are the right ones). Please let me know what else you need.

    Ivana

  • #1245

    ivca-cz
    Participant
  • #1303

    mspyhm2001
    Participant

    Dear Colleagues, I work on S1-TOPS mode. Please see attached zip file.
    I cant process all my area in any region. i am using compiled version 12-sep-2015 and trying for update to 27-oct-2015 but i cant connect to server. please help me

  • #1305

    mspyhm2001
    Participant
  • #1320

    periz
    Keymaster

    Hi,

    we experienced some problems related to the forum (missing replies, delays in notifications).
    Please try again to download the last sw version. If you fail it, write directly to the admin and you will receive the update

    dp

  • #1368

    ivca-cz
    Participant

    Hello,

    I am attaching an error I encountered more times (but sometimes not; in different areas), I am processing an area close to image border; what is the problem here and how shall I avoid it?

    If I understand it well, it concerns the number of bursts…?

    Thank you, Ivana

    Attachments:
    You must be logged in to view attached files.
    • #1370

      Yuxiao
      Moderator

      Hi Dear Ivana,

      Thanks for the testing, this is indeed a bug and I’ve fixed it.

      I’ll submit the fixed code for sarproz.

      Between now and the publishing of new version, as a temporarily work around, this bug can generally be avoided by not selecting areas close to the border of Sentinel image.

      Also in your log I see that you have sentinel images on the same data in adjacent areas. We are also developing code for merging adjacent tops images at this moment.

      Thank again for the feedback,

      Yuxiao

You must be logged in to reply to this topic.